<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Chapter 8 Optimization</title>
<link rel="stylesheet" href="mvl.css" type="text/css" />
<meta name="generator" content="DocBook XSL Stylesheets + chunker.py v1.9.2" />
<link rel="start" href="index.html" title="{book-title}" />
<link rel="up" href="" title="" />
<link rel="prev" href="backup-and-recovery.html" title="Chapter 7 Backup and Recovery" />
<link rel="next" href="language-structure.html" title="Chapter 9 Language Structure" />
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
<div class="navheader">
<table width="100%" summary="Navigation header">
<tr>
<th colspan="3" align="center">Chapter 8 Optimization</th>
</tr>
<tr>
<td width="20%" align="left"><a accesskey="p" href="backup-and-recovery.html">Prev</a> </td>
<th width="60%" align="center"></th>
<td width="20%" align="right"> <a accesskey="n" href="language-structure.html">Next</a></td>
</tr>
</table>
<hr>
</div>
<div class="chapter">
<div class="titlepage">
<div>
<div>
<h1 class="title"><a name="optimization"></a>Chapter 8 Optimization</h1>

</div>

</div>

</div>
<div class="toc">
<p><b>Table of Contents</b></p><dl class="toc"><dt><span class="section"><a href="optimization.html#optimize-overview">8.1 Optimization Overview</a></span></dt><dt><span class="section"><a href="optimization.html#statement-optimization">8.2 Optimizing SQL Statements</a></span></dt><dd><dl><dt><span class="section"><a href="optimization.html#select-optimization">8.2.1 Optimizing SELECT Statements</a></span></dt><dt><span class="section"><a href="optimization.html#subquery-optimization">8.2.2 Optimizing Subqueries, Derived Tables, View References, and Common Table
Expressions</a></span></dt><dt><span class="section"><a href="optimization.html#information-schema-optimization">8.2.3 Optimizing INFORMATION_SCHEMA Queries</a></span></dt><dt><span class="section"><a href="optimization.html#performance-schema-optimization">8.2.4 Optimizing Performance Schema Queries</a></span></dt><dt><span class="section"><a href="optimization.html#data-change-optimization">8.2.5 Optimizing Data Change Statements</a></span></dt><dt><span class="section"><a href="optimization.html#permission-optimization">8.2.6 Optimizing Database Privileges</a></span></dt><dt><span class="section"><a href="optimization.html#miscellaneous-optimization-tips">8.2.7 Other Optimization Tips</a></span></dt></dl></dd><dt><span class="section"><a href="optimization.html#optimization-indexes">8.3 Optimization and Indexes</a></span></dt><dd><dl><dt><span class="section"><a href="optimization.html#mysql-indexes">8.3.1 How MySQL Uses Indexes</a></span></dt><dt><span class="section"><a href="optimization.html#primary-key-optimization">8.3.2 Primary Key Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#spatial-index-optimization">8.3.3 SPATIAL Index Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#foreign-key-optimization">8.3.4 Foreign Key Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#column-indexes">8.3.5 Column Indexes</a></span></dt><dt><span class="section"><a href="optimization.html#multiple-column-indexes">8.3.6 Multiple-Column Indexes</a></span></dt><dt><span class="section"><a href="optimization.html#verifying-index-usage">8.3.7 Verifying Index Usage</a></span></dt><dt><span class="section"><a href="optimization.html#index-statistics">8.3.8 InnoDB and MyISAM Index Statistics Collection</a></span></dt><dt><span class="section"><a href="optimization.html#index-btree-hash">8.3.9 Comparison of B-Tree and Hash Indexes</a></span></dt><dt><span class="section"><a href="optimization.html#index-extensions">8.3.10 Use of Index Extensions</a></span></dt><dt><span class="section"><a href="optimization.html#generated-column-index-optimizations">8.3.11 Optimizer Use of Generated Column Indexes</a></span></dt><dt><span class="section"><a href="optimization.html#invisible-indexes">8.3.12 Invisible Indexes</a></span></dt><dt><span class="section"><a href="optimization.html#descending-indexes">8.3.13 Descending Indexes</a></span></dt><dt><span class="section"><a href="optimization.html#timestamp-lookups">8.3.14 Indexed Lookups from TIMESTAMP Columns</a></span></dt></dl></dd><dt><span class="section"><a href="optimization.html#optimizing-database-structure">8.4 Optimizing Database Structure</a></span></dt><dd><dl><dt><span class="section"><a href="optimization.html#data-size">8.4.1 Optimizing Data Size</a></span></dt><dt><span class="section"><a href="optimization.html#optimize-data-types">8.4.2 Optimizing MySQL Data Types</a></span></dt><dt><span class="section"><a href="optimization.html#optimize-multi-tables">8.4.3 Optimizing for Many Tables</a></span></dt><dt><span class="section"><a href="optimization.html#internal-temporary-tables">8.4.4 Internal Temporary Table Use in MySQL</a></span></dt><dt><span class="section"><a href="optimization.html#database-count-limit">8.4.5 Limits on Number of Databases and Tables</a></span></dt><dt><span class="section"><a href="optimization.html#table-size-limit">8.4.6 Limits on Table Size</a></span></dt><dt><span class="section"><a href="optimization.html#column-count-limit">8.4.7 Limits on Table Column Count and Row Size</a></span></dt></dl></dd><dt><span class="section"><a href="optimization.html#optimizing-innodb">8.5 Optimizing for InnoDB Tables</a></span></dt><dd><dl><dt><span class="section"><a href="optimization.html#optimizing-innodb-storage-layout">8.5.1 Optimizing Storage Layout for InnoDB Tables</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-innodb-transaction-management">8.5.2 Optimizing InnoDB Transaction Management</a></span></dt><dt><span class="section"><a href="optimization.html#innodb-performance-ro-txn">8.5.3 Optimizing InnoDB Read-Only Transactions</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-innodb-logging">8.5.4 Optimizing InnoDB Redo Logging</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-innodb-bulk-data-loading">8.5.5 Bulk Data Loading for InnoDB Tables</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-innodb-queries">8.5.6 Optimizing InnoDB Queries</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-innodb-ddl-operations">8.5.7 Optimizing InnoDB DDL Operations</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-innodb-diskio">8.5.8 Optimizing InnoDB Disk I/O</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-innodb-configuration-variables">8.5.9 Optimizing InnoDB Configuration Variables</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-innodb-many-tables">8.5.10 Optimizing InnoDB for Systems with Many Tables</a></span></dt></dl></dd><dt><span class="section"><a href="optimization.html#optimizing-myisam">8.6 Optimizing for MyISAM Tables</a></span></dt><dd><dl><dt><span class="section"><a href="optimization.html#optimizing-queries-myisam">8.6.1 Optimizing MyISAM Queries</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-myisam-bulk-data-loading">8.6.2 Bulk Data Loading for MyISAM Tables</a></span></dt><dt><span class="section"><a href="optimization.html#repair-table-optimization">8.6.3 Optimizing REPAIR TABLE Statements</a></span></dt></dl></dd><dt><span class="section"><a href="optimization.html#optimizing-memory-tables">8.7 Optimizing for MEMORY Tables</a></span></dt><dt><span class="section"><a href="optimization.html#execution-plan-information">8.8 Understanding the Query Execution Plan</a></span></dt><dd><dl><dt><span class="section"><a href="optimization.html#using-explain">8.8.1 Optimizing Queries with EXPLAIN</a></span></dt><dt><span class="section"><a href="optimization.html#explain-output">8.8.2 EXPLAIN Output Format</a></span></dt><dt><span class="section"><a href="optimization.html#explain-extended">8.8.3 Extended EXPLAIN Output Format</a></span></dt><dt><span class="section"><a href="optimization.html#explain-for-connection">8.8.4 Obtaining Execution Plan Information for a Named Connection</a></span></dt><dt><span class="section"><a href="optimization.html#estimating-performance">8.8.5 Estimating Query Performance</a></span></dt></dl></dd><dt><span class="section"><a href="optimization.html#controlling-optimizer">8.9 Controlling the Query Optimizer</a></span></dt><dd><dl><dt><span class="section"><a href="optimization.html#controlling-query-plan-evaluation">8.9.1 Controlling Query Plan Evaluation</a></span></dt><dt><span class="section"><a href="optimization.html#switchable-optimizations">8.9.2 Switchable Optimizations</a></span></dt><dt><span class="section"><a href="optimization.html#optimizer-hints">8.9.3 Optimizer Hints</a></span></dt><dt><span class="section"><a href="optimization.html#index-hints">8.9.4 Index Hints</a></span></dt><dt><span class="section"><a href="optimization.html#cost-model">8.9.5 The Optimizer Cost Model</a></span></dt><dt><span class="section"><a href="optimization.html#optimizer-statistics">8.9.6 Optimizer Statistics</a></span></dt></dl></dd><dt><span class="section"><a href="optimization.html#buffering-caching">8.10 Buffering and Caching</a></span></dt><dd><dl><dt><span class="section"><a href="optimization.html#innodb-buffer-pool-optimization">8.10.1 InnoDB Buffer Pool Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#myisam-key-cache">8.10.2 The MyISAM Key Cache</a></span></dt><dt><span class="section"><a href="optimization.html#statement-caching">8.10.3 Caching of Prepared Statements and Stored Programs</a></span></dt></dl></dd><dt><span class="section"><a href="optimization.html#locking-issues">8.11 Optimizing Locking Operations</a></span></dt><dd><dl><dt><span class="section"><a href="optimization.html#internal-locking">8.11.1 Internal Locking Methods</a></span></dt><dt><span class="section"><a href="optimization.html#table-locking">8.11.2 Table Locking Issues</a></span></dt><dt><span class="section"><a href="optimization.html#concurrent-inserts">8.11.3 Concurrent Inserts</a></span></dt><dt><span class="section"><a href="optimization.html#metadata-locking">8.11.4 Metadata Locking</a></span></dt><dt><span class="section"><a href="optimization.html#external-locking">8.11.5 External Locking</a></span></dt></dl></dd><dt><span class="section"><a href="optimization.html#optimizing-server">8.12 Optimizing the MySQL Server</a></span></dt><dd><dl><dt><span class="section"><a href="optimization.html#disk-issues">8.12.1 Optimizing Disk I/O</a></span></dt><dt><span class="section"><a href="optimization.html#symbolic-links">8.12.2 Using Symbolic Links</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-memory">8.12.3 Optimizing Memory Use</a></span></dt></dl></dd><dt><span class="section"><a href="optimization.html#optimize-benchmarking">8.13 Measuring Performance (Benchmarking)</a></span></dt><dd><dl><dt><span class="section"><a href="optimization.html#select-benchmarking">8.13.1 Measuring the Speed of Expressions and Functions</a></span></dt><dt><span class="section"><a href="optimization.html#custom-benchmarks">8.13.2 Using Your Own Benchmarks</a></span></dt><dt><span class="section"><a href="optimization.html#monitoring-performance-schema">8.13.3 Measuring Performance with performance_schema</a></span></dt></dl></dd><dt><span class="section"><a href="optimization.html#thread-information">8.14 Examining Thread Information</a></span></dt><dd><dl><dt><span class="section"><a href="optimization.html#thread-commands">8.14.1 Thread Command Values</a></span></dt><dt><span class="section"><a href="optimization.html#general-thread-states">8.14.2 General Thread States</a></span></dt><dt><span class="section"><a href="optimization.html#master-thread-states">8.14.3 Replication Master Thread States</a></span></dt><dt><span class="section"><a href="optimization.html#slave-io-thread-states">8.14.4 Replication Slave I/O Thread States</a></span></dt><dt><span class="section"><a href="optimization.html#slave-sql-thread-states">8.14.5 Replication Slave SQL Thread States</a></span></dt><dt><span class="section"><a href="optimization.html#slave-connection-thread-states">8.14.6 Replication Slave Connection Thread States</a></span></dt><dt><span class="section"><a href="optimization.html#mysql-cluster-thread-states">8.14.7 NDB Cluster Thread States</a></span></dt><dt><span class="section"><a href="optimization.html#event-scheduler-thread-states">8.14.8 Event Scheduler Thread States</a></span></dt></dl></dd></dl>
</div>
<a class="indexterm" name="idm46444353763520"></a><a class="indexterm" name="idm46444353762448"></a><a class="indexterm" name="idm46444353761376"></a><p>
    This chapter explains how to optimize MySQL performance and provides
    examples. Optimization involves configuring, tuning, and measuring
    performance, at several levels. Depending on your job role
    (developer, DBA, or a combination of both), you might optimize at
    the level of individual SQL statements, entire applications, a
    single database server, or multiple networked database servers.
    Sometimes you can be proactive and plan in advance for performance,
    while other times you might troubleshoot a configuration or code
    issue after a problem occurs. Optimizing CPU and memory usage can
    also improve scalability, allowing the database to handle more load
    without slowing down.
</p>
<div class="section">

<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a name="optimize-overview"></a>8.1 Optimization Overview</h2>
</div>
</div>
</div>
<p>
      Database performance depends on several factors at the database
      level, such as tables, queries, and configuration settings. These
      software constructs result in CPU and I/O operations at the
      hardware level, which you must minimize and make as efficient as
      possible. As you work on database performance, you start by
      learning the high-level rules and guidelines for the software
      side, and measuring performance using wall-clock time. As you
      become an expert, you learn more about what happens internally,
      and start measuring things such as CPU cycles and I/O operations.
    </p><p>
      Typical users aim to get the best database performance out of
      their existing software and hardware configurations. Advanced
      users look for opportunities to improve the MySQL software itself,
      or develop their own storage engines and hardware appliances to
      expand the MySQL ecosystem.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a class="xref" href="optimization.html#optimize-database-level" title="Optimizing at the Database Level">Optimizing at the Database Level</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#optimize-hardware-level" title="Optimizing at the Hardware Level">Optimizing at the Hardware Level</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#optimize-portability-performance" title="Balancing Portability and Performance">Balancing Portability and Performance</a></p></li></ul>
</div>

<div class="simplesect">

<div class="titlepage">
<div>

<div class="simple">
<h3 class="title"><a name="optimize-database-level"></a>Optimizing at the Database Level</h3>

</div>

</div>

</div>
<p>
        The most important factor in making a database application fast
        is its basic design:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Are the tables structured properly? In particular, do the
            columns have the right data types, and does each table have
            the appropriate columns for the type of work? For example,
            applications that perform frequent updates often have many
            tables with few columns, while applications that analyze
            large amounts of data often have few tables with many
            columns.
          </p></li><li class="listitem"><p>
            Are the right
            <a class="link" href="optimization.html#optimization-indexes" title="8.3 Optimization and Indexes">indexes</a> in place
            to make queries efficient?
          </p></li><li class="listitem"><p>
            Are you using the appropriate storage engine for each table,
            and taking advantage of the strengths and features of each
            storage engine you use? In particular, the choice of a
            transactional storage engine such as
            <code class="literal"><a class="link" href="optimization.html#optimizing-innodb" title="8.5 Optimizing for InnoDB Tables">InnoDB</a></code>
            or a nontransactional one such as
            <code class="literal"><a class="link" href="optimization.html#optimizing-myisam" title="8.6 Optimizing for MyISAM Tables">MyISAM</a></code>
            can be very important for performance and scalability.
</p>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
              <code class="literal">InnoDB</code> is the default storage engine
              for new tables. In practice, the advanced
              <code class="literal">InnoDB</code> performance features mean that
              <code class="literal">InnoDB</code> tables often outperform the
              simpler <code class="literal">MyISAM</code> tables, especially for a
              busy database.
</p>
</div>
</li><li class="listitem"><p>
            Does each table use an appropriate row format? This choice
            also depends on the storage engine used for the table. In
            particular, compressed tables use less disk space and so
            require less disk I/O to read and write the data.
            Compression is available for all kinds of workloads with
            <code class="literal">InnoDB</code> tables, and for read-only
            <code class="literal">MyISAM</code> tables.
          </p></li><li class="listitem"><p>
            Does the application use an appropriate
            <a class="link" href="optimization.html#locking-issues" title="8.11 Optimizing Locking Operations">locking strategy</a>? For
            example, by allowing shared access when possible so that
            database operations can run concurrently, and requesting
            exclusive access when appropriate so that critical
            operations get top priority. Again, the choice of storage
            engine is significant. The <code class="literal">InnoDB</code> storage
            engine handles most locking issues without involvement from
            you, allowing for better concurrency in the database and
            reducing the amount of experimentation and tuning for your
            code.
          </p></li><li class="listitem"><p>
            Are all <a class="link" href="optimization.html#buffering-caching" title="8.10 Buffering and Caching">memory areas used
            for caching</a> sized correctly? That is, large enough to
            hold frequently accessed data, but not so large that they
            overload physical memory and cause paging. The main memory
            areas to configure are the <code class="literal">InnoDB</code> buffer
            pool and the <code class="literal">MyISAM</code> key cache.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h3 class="title"><a name="optimize-hardware-level"></a>Optimizing at the Hardware Level</h3>

</div>

</div>

</div>
<p>
        Any database application eventually hits hardware limits as the
        database becomes more and more busy. A DBA must evaluate whether
        it is possible to tune the application or reconfigure the server
        to avoid these
        <a class="link" href="glossary.html#glos_bottleneck" title="bottleneck">bottlenecks</a>, or whether
        more hardware resources are required. System bottlenecks
        typically arise from these sources:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Disk seeks. It takes time for the disk to find a piece of
            data. With modern disks, the mean time for this is usually
            lower than 10ms, so we can in theory do about 100 seeks a
            second. This time improves slowly with new disks and is very
            hard to optimize for a single table. The way to optimize
            seek time is to distribute the data onto more than one disk.
          </p></li><li class="listitem"><p>
            Disk reading and writing. When the disk is at the correct
            position, we need to read or write the data. With modern
            disks, one disk delivers at least 10–20MB/s
            throughput. This is easier to optimize than seeks because
            you can read in parallel from multiple disks.
          </p></li><li class="listitem"><p>
            CPU cycles. When the data is in main memory, we must process
            it to get our result. Having large tables compared to the
            amount of memory is the most common limiting factor. But
            with small tables, speed is usually not the problem.
          </p></li><li class="listitem"><p>
            Memory bandwidth. When the CPU needs more data than can fit
            in the CPU cache, main memory bandwidth becomes a
            bottleneck. This is an uncommon bottleneck for most systems,
            but one to be aware of.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h3 class="title"><a name="optimize-portability-performance"></a>Balancing Portability and Performance</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444353721712"></a><p>
        To use performance-oriented SQL extensions in a portable MySQL
        program, you can wrap MySQL-specific keywords in a statement
        within <code class="literal">/*! */</code> comment delimiters. Other SQL
        servers ignore the commented keywords. For information about
        writing comments, see <a class="xref" href="language-structure.html#comments" title="9.6 Comment Syntax">Section 9.6, “Comment Syntax”</a>.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a name="statement-optimization"></a>8.2 Optimizing SQL Statements</h2>

</div>

</div>

</div>
<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#select-optimization">8.2.1 Optimizing SELECT Statements</a></span></dt><dt><span class="section"><a href="optimization.html#subquery-optimization">8.2.2 Optimizing Subqueries, Derived Tables, View References, and Common Table
Expressions</a></span></dt><dt><span class="section"><a href="optimization.html#information-schema-optimization">8.2.3 Optimizing INFORMATION_SCHEMA Queries</a></span></dt><dt><span class="section"><a href="optimization.html#performance-schema-optimization">8.2.4 Optimizing Performance Schema Queries</a></span></dt><dt><span class="section"><a href="optimization.html#data-change-optimization">8.2.5 Optimizing Data Change Statements</a></span></dt><dt><span class="section"><a href="optimization.html#permission-optimization">8.2.6 Optimizing Database Privileges</a></span></dt><dt><span class="section"><a href="optimization.html#miscellaneous-optimization-tips">8.2.7 Other Optimization Tips</a></span></dt></dl>
</div>
<a class="indexterm" name="idm46444353717184"></a><p>
      The core logic of a database application is performed through SQL
      statements, whether issued directly through an interpreter or
      submitted behind the scenes through an API. The tuning guidelines
      in this section help to speed up all kinds of MySQL applications.
      The guidelines cover SQL operations that read and write data, the
      behind-the-scenes overhead for SQL operations in general, and
      operations used in specific scenarios such as database monitoring.
</p>
<div class="section">

<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="select-optimization"></a>8.2.1 Optimizing SELECT Statements</h3>

</div>

</div>

</div>

<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#where-optimization">8.2.1.1 WHERE Clause Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#range-optimization">8.2.1.2 Range Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#index-merge-optimization">8.2.1.3 Index Merge Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#hash-joins">8.2.1.4 Hash Join Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#condition-pushdown-optimization">8.2.1.5 Engine Condition Pushdown Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#index-condition-pushdown-optimization">8.2.1.6 Index Condition Pushdown Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#nested-loop-joins">8.2.1.7 Nested-Loop Join Algorithms</a></span></dt><dt><span class="section"><a href="optimization.html#nested-join-optimization">8.2.1.8 Nested Join Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#outer-join-optimization">8.2.1.9 Outer Join Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#outer-join-simplification">8.2.1.10 Outer Join Simplification</a></span></dt><dt><span class="section"><a href="optimization.html#mrr-optimization">8.2.1.11 Multi-Range Read Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#bnl-bka-optimization">8.2.1.12 Block Nested-Loop and Batched Key Access Joins</a></span></dt><dt><span class="section"><a href="optimization.html#condition-filtering">8.2.1.13 Condition Filtering</a></span></dt><dt><span class="section"><a href="optimization.html#constant-folding-optimization">8.2.1.14 Constant-Folding Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#is-null-optimization">8.2.1.15 IS NULL Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#order-by-optimization">8.2.1.16 ORDER BY Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#group-by-optimization">8.2.1.17 GROUP BY Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#distinct-optimization">8.2.1.18 DISTINCT Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#limit-optimization">8.2.1.19 LIMIT Query Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#function-optimization">8.2.1.20 Function Call Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#window-function-optimization">8.2.1.21 Window Function Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#row-constructor-optimization">8.2.1.22 Row Constructor Expression Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#table-scan-avoidance">8.2.1.23 Avoiding Full Table Scans</a></span></dt></dl>
</div>
<a class="indexterm" name="idm46444353713696"></a><a class="indexterm" name="idm46444353712240"></a><a class="indexterm" name="idm46444353710752"></a><p>
        Queries, in the form of <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>
        statements, perform all the lookup operations in the database.
        Tuning these statements is a top priority, whether to achieve
        sub-second response times for dynamic web pages, or to chop
        hours off the time to generate huge overnight reports.
      </p><p>
        Besides <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements, the
        tuning techniques for queries also apply to constructs such as
        <a class="link" href="sql-statements.html#create-table-select" title="13.1.20.4 CREATE TABLE ... SELECT Statement"><code class="literal">CREATE
        TABLE...AS SELECT</code></a>,
        <a class="link" href="sql-statements.html#insert-select" title="13.2.6.1 INSERT ... SELECT Statement"><code class="literal">INSERT
        INTO...SELECT</code></a>, and <code class="literal">WHERE</code> clauses in
        <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a> statements. Those
        statements have additional performance considerations because
        they combine write operations with the read-oriented query
        operations.
      </p><p>
        NDB Cluster supports a join pushdown optimization whereby a
        qualifying join is sent in its entirety to NDB Cluster data
        nodes, where it can be distributed among them and executed in
        parallel. For more information about this optimization, see
        <a class="xref" href="mysql-cluster.html#ndb_join_pushdown-conditions" title="Conditions for NDB pushdown joins">Conditions for NDB pushdown joins</a>.
      </p><p>
        The main considerations for optimizing queries are:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            To make a slow <code class="literal">SELECT ... WHERE</code> query
            faster, the first thing to check is whether you can add an
            <a class="link" href="glossary.html#glos_index" title="index">index</a>. Set up indexes on
            columns used in the <code class="literal">WHERE</code> clause, to
            speed up evaluation, filtering, and the final retrieval of
            results. To avoid wasted disk space, construct a small set
            of indexes that speed up many related queries used in your
            application.
          </p><p>
            Indexes are especially important for queries that reference
            different tables, using features such as
            <a class="link" href="glossary.html#glos_join" title="join">joins</a> and
            <a class="link" href="glossary.html#glos_foreign_key" title="foreign key">foreign keys</a>. You
            can use the <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> statement
            to determine which indexes are used for a
            <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>. See
            <a class="xref" href="optimization.html#mysql-indexes" title="8.3.1 How MySQL Uses Indexes">Section 8.3.1, “How MySQL Uses Indexes”</a> and
            <a class="xref" href="optimization.html#using-explain" title="8.8.1 Optimizing Queries with EXPLAIN">Section 8.8.1, “Optimizing Queries with EXPLAIN”</a>.
          </p></li><li class="listitem"><p>
            Isolate and tune any part of the query, such as a function
            call, that takes excessive time. Depending on how the query
            is structured, a function could be called once for every row
            in the result set, or even once for every row in the table,
            greatly magnifying any inefficiency.
          </p></li><li class="listitem"><p>
            Minimize the number of
            <a class="link" href="glossary.html#glos_full_table_scan" title="full table scan">full table scans</a>
            in your queries, particularly for big tables.
          </p></li><li class="listitem"><p>
            Keep table statistics up to date by using the
            <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE TABLE</code></a> statement
            periodically, so the optimizer has the information needed to
            construct an efficient execution plan.
          </p></li><li class="listitem"><p>
            Learn the tuning techniques, indexing techniques, and
            configuration parameters that are specific to the storage
            engine for each table. Both <code class="literal">InnoDB</code> and
            <code class="literal">MyISAM</code> have sets of guidelines for
            enabling and sustaining high performance in queries. For
            details, see <a class="xref" href="optimization.html#optimizing-innodb-queries" title="8.5.6 Optimizing InnoDB Queries">Section 8.5.6, “Optimizing InnoDB Queries”</a> and
            <a class="xref" href="optimization.html#optimizing-queries-myisam" title="8.6.1 Optimizing MyISAM Queries">Section 8.6.1, “Optimizing MyISAM Queries”</a>.
          </p></li><li class="listitem"><p>
            You can optimize single-query transactions for
            <code class="literal">InnoDB</code> tables, using the technique in
            <a class="xref" href="optimization.html#innodb-performance-ro-txn" title="8.5.3 Optimizing InnoDB Read-Only Transactions">Section 8.5.3, “Optimizing InnoDB Read-Only Transactions”</a>.
          </p></li><li class="listitem"><p>
            Avoid transforming the query in ways that make it hard to
            understand, especially if the optimizer does some of the
            same transformations automatically.
          </p></li><li class="listitem"><p>
            If a performance issue is not easily solved by one of the
            basic guidelines, investigate the internal details of the
            specific query by reading the
            <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> plan and adjusting
            your indexes, <code class="literal">WHERE</code> clauses, join
            clauses, and so on. (When you reach a certain level of
            expertise, reading the
            <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> plan might be your
            first step for every query.)
          </p></li><li class="listitem"><p>
            Adjust the size and properties of the memory areas that
            MySQL uses for caching. With efficient use of the
            <code class="literal">InnoDB</code>
            <a class="link" href="glossary.html#glos_buffer_pool" title="buffer pool">buffer pool</a>,
            <code class="literal">MyISAM</code> key cache, and the MySQL query
            cache, repeated queries run faster because the results are
            retrieved from memory the second and subsequent times.
          </p></li><li class="listitem"><p>
            Even for a query that runs fast using the cache memory
            areas, you might still optimize further so that they require
            less cache memory, making your application more scalable.
            Scalability means that your application can handle more
            simultaneous users, larger requests, and so on without
            experiencing a big drop in performance.
          </p></li><li class="listitem"><p>
            Deal with locking issues, where the speed of your query
            might be affected by other sessions accessing the tables at
            the same time.
</p></li></ul>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="where-optimization"></a>8.2.1.1 WHERE Clause Optimization</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444353664432"></a><a class="indexterm" name="idm46444353663392"></a><p>
          This section discusses optimizations that can be made for
          processing <code class="literal">WHERE</code> clauses. The examples use
          <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements, but the same
          optimizations apply for <code class="literal">WHERE</code> clauses in
          <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a> and
          <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a> statements.
</p>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
            Because work on the MySQL optimizer is ongoing, not all of
            the optimizations that MySQL performs are documented here.
</p>
</div>
<p>
          You might be tempted to rewrite your queries to make
          arithmetic operations faster, while sacrificing readability.
          Because MySQL does similar optimizations automatically, you
          can often avoid this work, and leave the query in a more
          understandable and maintainable form. Some of the
          optimizations performed by MySQL follow:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Removal of unnecessary parentheses:
            </p><pre data-lang="sql" class="programlisting">   ((a AND b) AND c OR (((a AND b) AND (c AND d))))
-&gt; (a AND b AND c) OR (a AND b AND c AND d)</pre></li><li class="listitem"><p>
              Constant folding:
            </p><pre data-lang="sql" class="programlisting">   (a&lt;b AND b=c) AND a=5
-&gt; b&gt;5 AND b=c AND a=5</pre></li><li class="listitem"><p>
              Constant condition removal:
            </p><pre data-lang="sql" class="programlisting">   (b&gt;=5 AND b=5) OR (b=6 AND 5=5) OR (b=7 AND 5=6)
-&gt; b=5 OR b=6</pre><p>
              In MySQL 8.0.14 and later, this takes place during
              preparation rather than during the optimization phase,
              which helps in simplification of joins. See
              <a class="xref" href="optimization.html#outer-join-optimization" title="8.2.1.9 Outer Join Optimization">Section 8.2.1.9, “Outer Join Optimization”</a>, for further
              information and examples.
            </p></li><li class="listitem"><p>
              Constant expressions used by indexes are evaluated only
              once.
            </p></li><li class="listitem"><p>
              Beginning with MySQL 8.0.16, comparisons of columns of
              numeric types with constant values are checked and folded
              or removed for invalid or out-of-rage values:
            </p><pre data-lang="sql" class="programlisting"># CREATE TABLE t (c TINYINT UNSIGNED NOT NULL);
  SELECT * FROM t WHERE c ≪ 256;
-≫ SELECT * FROM t WHERE 1;</pre><p>
              See <a class="xref" href="optimization.html#constant-folding-optimization" title="8.2.1.14 Constant-Folding Optimization">Section 8.2.1.14, “Constant-Folding Optimization”</a>, for
              more information.
            </p></li><li class="listitem"><p>
              <a class="link" href="functions.html#function_count"><code class="literal">COUNT(*)</code></a> on a single table
              without a <code class="literal">WHERE</code> is retrieved directly
              from the table information for <code class="literal">MyISAM</code>
              and <code class="literal">MEMORY</code> tables. This is also done
              for any <code class="literal">NOT NULL</code> expression when used
              with only one table.
            </p></li><li class="listitem"><p>
              Early detection of invalid constant expressions. MySQL
              quickly detects that some
              <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements are
              impossible and returns no rows.
            </p></li><li class="listitem"><p>
              <code class="literal">HAVING</code> is merged with
              <code class="literal">WHERE</code> if you do not use <code class="literal">GROUP
              BY</code> or aggregate functions
              (<a class="link" href="functions.html#function_count"><code class="literal">COUNT()</code></a>,
              <a class="link" href="functions.html#function_min"><code class="literal">MIN()</code></a>, and so on).
            </p></li><li class="listitem"><p>
              For each table in a join, a simpler
              <code class="literal">WHERE</code> is constructed to get a fast
              <code class="literal">WHERE</code> evaluation for the table and also
              to skip rows as soon as possible.
            </p></li><li class="listitem"><p>
              <a class="indexterm" name="idm46444353627584"></a>

              <a class="indexterm" name="idm46444353626512"></a>

              All constant tables are read first before any other tables
              in the query. A constant table is any of the following:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                  An empty table or a table with one row.
                </p></li><li class="listitem"><p>
                  A table that is used with a <code class="literal">WHERE</code>
                  clause on a <code class="literal">PRIMARY KEY</code> or a
                  <code class="literal">UNIQUE</code> index, where all index parts
                  are compared to constant expressions and are defined
                  as <code class="literal">NOT NULL</code>.
</p></li></ul>
</div>
<p>
              All of the following tables are used as constant tables:
            </p><pre data-lang="sql" class="programlisting">SELECT * FROM t WHERE <em class="replaceable"><code>primary_key</code></em>=1;
SELECT * FROM t1,t2
  WHERE t1.<em class="replaceable"><code>primary_key</code></em>=1 AND t2.<em class="replaceable"><code>primary_key</code></em>=t1.id;
</pre></li><li class="listitem"><p>
              The best join combination for joining the tables is found
              by trying all possibilities. If all columns in
              <code class="literal">ORDER BY</code> and <code class="literal">GROUP
              BY</code> clauses come from the same table, that table
              is preferred first when joining.
            </p></li><li class="listitem"><p>
              If there is an <code class="literal">ORDER BY</code> clause and a
              different <code class="literal">GROUP BY</code> clause, or if the
              <code class="literal">ORDER BY</code> or <code class="literal">GROUP BY</code>
              contains columns from tables other than the first table in
              the join queue, a temporary table is created.
            </p></li><li class="listitem"><p>
              If you use the <code class="literal">SQL_SMALL_RESULT</code>
              modifier, MySQL uses an in-memory temporary table.
            </p></li><li class="listitem"><p>
              Each table index is queried, and the best index is used
              unless the optimizer believes that it is more efficient to
              use a table scan. At one time, a scan was used based on
              whether the best index spanned more than 30% of the table,
              but a fixed percentage no longer determines the choice
              between using an index or a scan. The optimizer now is
              more complex and bases its estimate on additional factors
              such as table size, number of rows, and I/O block size.
            </p></li><li class="listitem"><p>
              In some cases, MySQL can read rows from the index without
              even consulting the data file. If all columns used from
              the index are numeric, only the index tree is used to
              resolve the query.
            </p></li><li class="listitem"><p>
              Before each row is output, those that do not match the
              <code class="literal">HAVING</code> clause are skipped.
</p></li></ul>
</div>
<p>
          Some examples of queries that are very fast:
        </p><pre data-lang="sql" class="programlisting">SELECT COUNT(*) FROM <em class="replaceable"><code>tbl_name</code></em>;

SELECT MIN(<em class="replaceable"><code>key_part1</code></em>),MAX(<em class="replaceable"><code>key_part1</code></em>) FROM <em class="replaceable"><code>tbl_name</code></em>;

SELECT MAX(<em class="replaceable"><code>key_part2</code></em>) FROM <em class="replaceable"><code>tbl_name</code></em>
  WHERE <em class="replaceable"><code>key_part1</code></em>=<em class="replaceable"><code>constant</code></em>;

SELECT ... FROM <em class="replaceable"><code>tbl_name</code></em>
  ORDER BY <em class="replaceable"><code>key_part1</code></em>,<em class="replaceable"><code>key_part2</code></em>,... LIMIT 10;

SELECT ... FROM <em class="replaceable"><code>tbl_name</code></em>
  ORDER BY <em class="replaceable"><code>key_part1</code></em> DESC, <em class="replaceable"><code>key_part2</code></em> DESC, ... LIMIT 10;
</pre><p>
          MySQL resolves the following queries using only the index
          tree, assuming that the indexed columns are numeric:
        </p><pre data-lang="sql" class="programlisting">SELECT <em class="replaceable"><code>key_part1</code></em>,<em class="replaceable"><code>key_part2</code></em> FROM <em class="replaceable"><code>tbl_name</code></em> WHERE <em class="replaceable"><code>key_part1</code></em>=<em class="replaceable"><code>val</code></em>;

SELECT COUNT(*) FROM <em class="replaceable"><code>tbl_name</code></em>
  WHERE <em class="replaceable"><code>key_part1</code></em>=<em class="replaceable"><code>val1</code></em> AND <em class="replaceable"><code>key_part2</code></em>=<em class="replaceable"><code>val2</code></em>;

SELECT <em class="replaceable"><code>key_part2</code></em> FROM <em class="replaceable"><code>tbl_name</code></em> GROUP BY <em class="replaceable"><code>key_part1</code></em>;
</pre><p>
          The following queries use indexing to retrieve the rows in
          sorted order without a separate sorting pass:
        </p><pre data-lang="sql" class="programlisting">SELECT ... FROM <em class="replaceable"><code>tbl_name</code></em>
  ORDER BY <em class="replaceable"><code>key_part1</code></em>,<em class="replaceable"><code>key_part2</code></em>,... ;

SELECT ... FROM <em class="replaceable"><code>tbl_name</code></em>
  ORDER BY <em class="replaceable"><code>key_part1</code></em> DESC, <em class="replaceable"><code>key_part2</code></em> DESC, ... ;
</pre>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="range-optimization"></a>8.2.1.2 Range Optimization</h4>

</div>

</div>

</div>
<p>
          The <a class="link" href="optimization.html#jointype_range"><code class="literal">range</code></a> access method
          uses a single index to retrieve a subset of table rows that
          are contained within one or several index value intervals. It
          can be used for a single-part or multiple-part index. The
          following sections describe conditions under which the
          optimizer uses range access.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a class="xref" href="optimization.html#range-access-single-part" title="Range Access Method for Single-Part Indexes">Range Access Method for Single-Part Indexes</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#range-access-multi-part" title="Range Access Method for Multiple-Part Indexes">Range Access Method for Multiple-Part Indexes</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#equality-range-optimization" title="Equality Range Optimization of Many-Valued Comparisons">Equality Range Optimization of Many-Valued Comparisons</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#range-access-skip-scan" title="Skip Scan Range Access Method">Skip Scan Range Access Method</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#row-constructor-range-optimization" title="Range Optimization of Row Constructor Expressions">Range Optimization of Row Constructor Expressions</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#range-optimization-memory-use" title="Limiting Memory Use for Range Optimization">Limiting Memory Use for Range Optimization</a></p></li></ul>
</div>

<div class="simplesect">

<div class="titlepage">
<div>

<div class="simple">
<h5 class="title"><a name="range-access-single-part"></a>Range Access Method for Single-Part Indexes</h5>

</div>

</div>

</div>
<p>
            For a single-part index, index value intervals can be
            conveniently represented by corresponding conditions in the
            <code class="literal">WHERE</code> clause, denoted as
            <span class="firstterm">range conditions</span>
            rather than <span class="quote">“<span class="quote">intervals.</span>”</span>
          </p><p>
            The definition of a range condition for a single-part index
            is as follows:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                For both <code class="literal">BTREE</code> and
                <code class="literal">HASH</code> indexes, comparison of a key
                part with a constant value is a range condition when
                using the
                <a class="link" href="functions.html#operator_equal"><code class="literal">=</code></a>,
                <a class="link" href="functions.html#operator_equal-to"><code class="literal">&lt;=&gt;</code></a>,
                <a class="link" href="functions.html#operator_in"><code class="literal">IN()</code></a>, <a class="link" href="functions.html#operator_is-null"><code class="literal">IS
                NULL</code></a>, or <a class="link" href="functions.html#operator_is-not-null"><code class="literal">IS NOT
                NULL</code></a> operators.
              </p></li><li class="listitem"><p>
                Additionally, for <code class="literal">BTREE</code> indexes,
                comparison of a key part with a constant value is a
                range condition when using the
                <a class="link" href="functions.html#operator_greater-than"><code class="literal">&gt;</code></a>,
                <a class="link" href="functions.html#operator_less-than"><code class="literal">&lt;</code></a>,
                <a class="link" href="functions.html#operator_greater-than-or-equal"><code class="literal">&gt;=</code></a>,
                <a class="link" href="functions.html#operator_less-than-or-equal"><code class="literal">&lt;=</code></a>,
                <a class="link" href="functions.html#operator_between"><code class="literal">BETWEEN</code></a>,
                <a class="link" href="functions.html#operator_not-equal"><code class="literal">!=</code></a>,
                or
                <a class="link" href="functions.html#operator_not-equal"><code class="literal">&lt;&gt;</code></a>
                operators, or <a class="link" href="functions.html#operator_like"><code class="literal">LIKE</code></a>
                comparisons if the argument to
                <a class="link" href="functions.html#operator_like"><code class="literal">LIKE</code></a> is a constant string
                that does not start with a wildcard character.
              </p></li><li class="listitem"><p>
                For all index types, multiple range conditions combined
                with <a class="link" href="functions.html#operator_or"><code class="literal">OR</code></a> or
                <a class="link" href="functions.html#operator_and"><code class="literal">AND</code></a> form a range condition.
</p></li></ul>
</div>
<p>
            <span class="quote">“<span class="quote">Constant value</span>”</span> in the preceding descriptions
            means one of the following:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                A constant from the query string
              </p></li><li class="listitem"><p>
                A column of a <a class="link" href="optimization.html#jointype_const"><code class="literal">const</code></a>
                or <a class="link" href="optimization.html#jointype_system"><code class="literal">system</code></a> table from
                the same join
              </p></li><li class="listitem"><p>
                The result of an uncorrelated subquery
              </p></li><li class="listitem"><p>
                Any expression composed entirely from subexpressions of
                the preceding types
</p></li></ul>
</div>
<p>
            Here are some examples of queries with range conditions in
            the <code class="literal">WHERE</code> clause:
          </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1
  WHERE <em class="replaceable"><code>key_col</code></em> &gt; 1
  AND <em class="replaceable"><code>key_col</code></em> &lt; 10;

SELECT * FROM t1
  WHERE <em class="replaceable"><code>key_col</code></em> = 1
  OR <em class="replaceable"><code>key_col</code></em> IN (15,18,20);

SELECT * FROM t1
  WHERE <em class="replaceable"><code>key_col</code></em> LIKE 'ab%'
  OR <em class="replaceable"><code>key_col</code></em> BETWEEN 'bar' AND 'foo';
</pre><p>
            Some nonconstant values may be converted to constants during
            the optimizer constant propagation phase.
          </p><p>
            MySQL tries to extract range conditions from the
            <code class="literal">WHERE</code> clause for each of the possible
            indexes. During the extraction process, conditions that
            cannot be used for constructing the range condition are
            dropped, conditions that produce overlapping ranges are
            combined, and conditions that produce empty ranges are
            removed.
          </p><p>
            Consider the following statement, where
            <code class="literal">key1</code> is an indexed column and
            <code class="literal">nonkey</code> is not indexed:
          </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 WHERE
  (key1 &lt; 'abc' AND (key1 LIKE 'abcde%' OR key1 LIKE '%b')) OR
  (key1 &lt; 'bar' AND nonkey = 4) OR
  (key1 &lt; 'uux' AND key1 &gt; 'z');</pre><p>
            The extraction process for key <code class="literal">key1</code> is as
            follows:
</p>
<div class="orderedlist">
<ol class="orderedlist" type="1"><li class="listitem"><p>
                Start with original <code class="literal">WHERE</code> clause:
              </p><pre data-lang="sql" class="programlisting">(key1 &lt; 'abc' AND (key1 LIKE 'abcde%' OR key1 LIKE '%b')) OR
(key1 &lt; 'bar' AND nonkey = 4) OR
(key1 &lt; 'uux' AND key1 &gt; 'z')</pre></li><li class="listitem"><p>
                Remove <code class="literal">nonkey = 4</code> and <code class="literal">key1
                LIKE '%b'</code> because they cannot be used for a
                range scan. The correct way to remove them is to replace
                them with <code class="literal">TRUE</code>, so that we do not
                miss any matching rows when doing the range scan.
                Replacing them with <code class="literal">TRUE</code> yields:
              </p><pre data-lang="sql" class="programlisting">(key1 &lt; 'abc' AND (key1 LIKE 'abcde%' OR TRUE)) OR
(key1 &lt; 'bar' AND TRUE) OR
(key1 &lt; 'uux' AND key1 &gt; 'z')</pre></li><li class="listitem"><p>
                Collapse conditions that are always true or false:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                    <code class="literal">(key1 LIKE 'abcde%' OR TRUE)</code> is
                    always true
                  </p></li><li class="listitem"><p>
                    <code class="literal">(key1 &lt; 'uux' AND key1 &gt;
                    'z')</code> is always false
</p></li></ul>
</div>
<p>
                Replacing these conditions with constants yields:
              </p><pre data-lang="clike" class="programlisting">(key1 &lt; 'abc' AND TRUE) OR (key1 &lt; 'bar' AND TRUE) OR (FALSE)</pre><p>
                Removing unnecessary <code class="literal">TRUE</code> and
                <code class="literal">FALSE</code> constants yields:
              </p><pre data-lang="clike" class="programlisting">(key1 &lt; 'abc') OR (key1 &lt; 'bar')</pre></li><li class="listitem"><p>
                Combining overlapping intervals into one yields the
                final condition to be used for the range scan:
</p><pre data-lang="clike" class="programlisting">(key1 &lt; 'bar')</pre></li></ol>
</div>
<p>
            In general (and as demonstrated by the preceding example),
            the condition used for a range scan is less restrictive than
            the <code class="literal">WHERE</code> clause. MySQL performs an
            additional check to filter out rows that satisfy the range
            condition but not the full <code class="literal">WHERE</code> clause.
          </p><p>
            The range condition extraction algorithm can handle nested
            <a class="link" href="functions.html#operator_and"><code class="literal">AND</code></a>/<a class="link" href="functions.html#operator_or"><code class="literal">OR</code></a>
            constructs of arbitrary depth, and its output does not
            depend on the order in which conditions appear in
            <code class="literal">WHERE</code> clause.
          </p><p>
            MySQL does not support merging multiple ranges for the
            <a class="link" href="optimization.html#jointype_range"><code class="literal">range</code></a> access method for
            spatial indexes. To work around this limitation, you can use
            a <a class="link" href="sql-statements.html#union" title="13.2.10.3 UNION Clause"><code class="literal">UNION</code></a> with identical
            <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements, except
            that you put each spatial predicate in a different
            <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h5 class="title"><a name="range-access-multi-part"></a>Range Access Method for Multiple-Part Indexes</h5>

</div>

</div>

</div>
<p>
            Range conditions on a multiple-part index are an extension
            of range conditions for a single-part index. A range
            condition on a multiple-part index restricts index rows to
            lie within one or several key tuple intervals. Key tuple
            intervals are defined over a set of key tuples, using
            ordering from the index.
          </p><p>
            For example, consider a multiple-part index defined as
            <code class="literal">key1(<em class="replaceable"><code>key_part1</code></em>,
            <em class="replaceable"><code>key_part2</code></em>,
            <em class="replaceable"><code>key_part3</code></em>)</code>, and the
            following set of key tuples listed in key order:
          </p><pre data-lang="clike" class="programlisting"><em class="replaceable"><code>key_part1</code></em>  <em class="replaceable"><code>key_part2</code></em>  <em class="replaceable"><code>key_part3</code></em>
  NULL       1          'abc'
  NULL       1          'xyz'
  NULL       2          'foo'
   1         1          'abc'
   1         1          'xyz'
   1         2          'abc'
   2         1          'aaa'
</pre><p>
            The condition <code class="literal"><em class="replaceable"><code>key_part1</code></em>
            = 1</code> defines this interval:
          </p><pre data-lang="clike" class="programlisting">(1,-inf,-inf) &lt;= (<em class="replaceable"><code>key_part1</code></em>,<em class="replaceable"><code>key_part2</code></em>,<em class="replaceable"><code>key_part3</code></em>) &lt; (1,+inf,+inf)
</pre><p>
            The interval covers the 4th, 5th, and 6th tuples in the
            preceding data set and can be used by the range access
            method.
          </p><p>
            By contrast, the condition
            <code class="literal"><em class="replaceable"><code>key_part3</code></em> =
            'abc'</code> does not define a single interval and cannot
            be used by the range access method.
          </p><p>
            The following descriptions indicate how range conditions
            work for multiple-part indexes in greater detail.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                For <code class="literal">HASH</code> indexes, each interval
                containing identical values can be used. This means that
                the interval can be produced only for conditions in the
                following form:
              </p><pre data-lang="sql" class="programlisting">    <em class="replaceable"><code>key_part1</code></em> <em class="replaceable"><code>cmp</code></em> <em class="replaceable"><code>const1</code></em>
AND <em class="replaceable"><code>key_part2</code></em> <em class="replaceable"><code>cmp</code></em> <em class="replaceable"><code>const2</code></em>
AND ...
AND <em class="replaceable"><code>key_partN</code></em> <em class="replaceable"><code>cmp</code></em> <em class="replaceable"><code>constN</code></em>;
</pre><p>
                Here, <em class="replaceable"><code>const1</code></em>,
                <em class="replaceable"><code>const2</code></em>, … are
                constants, <em class="replaceable"><code>cmp</code></em> is one of the
                <a class="link" href="functions.html#operator_equal"><code class="literal">=</code></a>,
                <a class="link" href="functions.html#operator_equal-to"><code class="literal">&lt;=&gt;</code></a>,
                or <a class="link" href="functions.html#operator_is-null"><code class="literal">IS NULL</code></a> comparison
                operators, and the conditions cover all index parts.
                (That is, there are <em class="replaceable"><code>N</code></em>
                conditions, one for each part of an
                <em class="replaceable"><code>N</code></em>-part index.) For example,
                the following is a range condition for a three-part
                <code class="literal">HASH</code> index:
              </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>key_part1</code></em> = 1 AND <em class="replaceable"><code>key_part2</code></em> IS NULL AND <em class="replaceable"><code>key_part3</code></em> = 'foo'
</pre><p>
                For the definition of what is considered to be a
                constant, see
                <a class="xref" href="optimization.html#range-access-single-part" title="Range Access Method for Single-Part Indexes">Range Access Method for Single-Part Indexes</a>.
              </p></li><li class="listitem"><p>
                For a <code class="literal">BTREE</code> index, an interval might
                be usable for conditions combined with
                <a class="link" href="functions.html#operator_and"><code class="literal">AND</code></a>, where each condition
                compares a key part with a constant value using
                <a class="link" href="functions.html#operator_equal"><code class="literal">=</code></a>,
                <a class="link" href="functions.html#operator_equal-to"><code class="literal">&lt;=&gt;</code></a>,
                <a class="link" href="functions.html#operator_is-null"><code class="literal">IS NULL</code></a>,
                <a class="link" href="functions.html#operator_greater-than"><code class="literal">&gt;</code></a>,
                <a class="link" href="functions.html#operator_less-than"><code class="literal">&lt;</code></a>,
                <a class="link" href="functions.html#operator_greater-than-or-equal"><code class="literal">&gt;=</code></a>,
                <a class="link" href="functions.html#operator_less-than-or-equal"><code class="literal">&lt;=</code></a>,
                <a class="link" href="functions.html#operator_not-equal"><code class="literal">!=</code></a>,
                <a class="link" href="functions.html#operator_not-equal"><code class="literal">&lt;&gt;</code></a>,
                <a class="link" href="functions.html#operator_between"><code class="literal">BETWEEN</code></a>, or
                <a class="link" href="functions.html#operator_like"><code class="literal">LIKE
                '<em class="replaceable"><code>pattern</code></em>'</code></a> (where
                <code class="literal">'<em class="replaceable"><code>pattern</code></em>'</code>
                does not start with a wildcard). An interval can be used
                as long as it is possible to determine a single key
                tuple containing all rows that match the condition (or
                two intervals if
                <a class="link" href="functions.html#operator_not-equal"><code class="literal">&lt;&gt;</code></a>
                or <a class="link" href="functions.html#operator_not-equal"><code class="literal">!=</code></a>
                is used).
              </p><p>
                The optimizer attempts to use additional key parts to
                determine the interval as long as the comparison
                operator is
                <a class="link" href="functions.html#operator_equal"><code class="literal">=</code></a>,
                <a class="link" href="functions.html#operator_equal-to"><code class="literal">&lt;=&gt;</code></a>,
                or <a class="link" href="functions.html#operator_is-null"><code class="literal">IS NULL</code></a>. If the operator
                is
                <a class="link" href="functions.html#operator_greater-than"><code class="literal">&gt;</code></a>,
                <a class="link" href="functions.html#operator_less-than"><code class="literal">&lt;</code></a>,
                <a class="link" href="functions.html#operator_greater-than-or-equal"><code class="literal">&gt;=</code></a>,
                <a class="link" href="functions.html#operator_less-than-or-equal"><code class="literal">&lt;=</code></a>,
                <a class="link" href="functions.html#operator_not-equal"><code class="literal">!=</code></a>,
                <a class="link" href="functions.html#operator_not-equal"><code class="literal">&lt;&gt;</code></a>,
                <a class="link" href="functions.html#operator_between"><code class="literal">BETWEEN</code></a>, or
                <a class="link" href="functions.html#operator_like"><code class="literal">LIKE</code></a>, the
                optimizer uses it but considers no more key parts. For
                the following expression, the optimizer uses
                <a class="link" href="functions.html#operator_equal"><code class="literal">=</code></a> from
                the first comparison. It also uses
                <a class="link" href="functions.html#operator_greater-than-or-equal"><code class="literal">&gt;=</code></a>
                from the second comparison but considers no further key
                parts and does not use the third comparison for interval
                construction:
              </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>key_part1</code></em> = 'foo' AND <em class="replaceable"><code>key_part2</code></em> &gt;= 10 AND <em class="replaceable"><code>key_part3</code></em> &gt; 10
</pre><p>
                The single interval is:
              </p><pre data-lang="sql" class="programlisting">('foo',10,-inf) &lt; (<em class="replaceable"><code>key_part1</code></em>,<em class="replaceable"><code>key_part2</code></em>,<em class="replaceable"><code>key_part3</code></em>) &lt; ('foo',+inf,+inf)
</pre><p>
                It is possible that the created interval contains more
                rows than the initial condition. For example, the
                preceding interval includes the value <code class="literal">('foo',
                11, 0)</code>, which does not satisfy the original
                condition.
              </p></li><li class="listitem"><p>
                If conditions that cover sets of rows contained within
                intervals are combined with
                <a class="link" href="functions.html#operator_or"><code class="literal">OR</code></a>, they form a condition
                that covers a set of rows contained within the union of
                their intervals. If the conditions are combined with
                <a class="link" href="functions.html#operator_and"><code class="literal">AND</code></a>, they form a condition
                that covers a set of rows contained within the
                intersection of their intervals. For example, for this
                condition on a two-part index:
              </p><pre data-lang="sql" class="programlisting">(<em class="replaceable"><code>key_part1</code></em> = 1 AND <em class="replaceable"><code>key_part2</code></em> &lt; 2) OR (<em class="replaceable"><code>key_part1</code></em> &gt; 5)
</pre><p>
                The intervals are:
              </p><pre data-lang="sql" class="programlisting">(1,-inf) &lt; (<em class="replaceable"><code>key_part1</code></em>,<em class="replaceable"><code>key_part2</code></em>) &lt; (1,2)
(5,-inf) &lt; (<em class="replaceable"><code>key_part1</code></em>,<em class="replaceable"><code>key_part2</code></em>)
</pre><p>
                In this example, the interval on the first line uses one
                key part for the left bound and two key parts for the
                right bound. The interval on the second line uses only
                one key part. The <code class="literal">key_len</code> column in
                the <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output
                indicates the maximum length of the key prefix used.
              </p><p>
                In some cases, <code class="literal">key_len</code> may indicate
                that a key part was used, but that might be not what you
                would expect. Suppose that
                <em class="replaceable"><code>key_part1</code></em> and
                <em class="replaceable"><code>key_part2</code></em> can be
                <code class="literal">NULL</code>. Then the
                <code class="literal">key_len</code> column displays two key part
                lengths for the following condition:
              </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>key_part1</code></em> &gt;= 1 AND <em class="replaceable"><code>key_part2</code></em> &lt; 2
</pre><p>
                But, in fact, the condition is converted to this:
              </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>key_part1</code></em> &gt;= 1 AND <em class="replaceable"><code>key_part2</code></em> IS NOT NULL
</pre></li></ul>
</div>
<p>
            For a description of how optimizations are performed to
            combine or eliminate intervals for range conditions on a
            single-part index, see
            <a class="xref" href="optimization.html#range-access-single-part" title="Range Access Method for Single-Part Indexes">Range Access Method for Single-Part Indexes</a>. Analogous steps
            are performed for range conditions on multiple-part indexes.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h5 class="title"><a name="equality-range-optimization"></a>Equality Range Optimization of Many-Valued Comparisons</h5>

</div>

</div>

</div>
<p>
            Consider these expressions, where
            <em class="replaceable"><code>col_name</code></em> is an indexed column:
          </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>col_name</code></em> IN(<em class="replaceable"><code>val1</code></em>, ..., <em class="replaceable"><code>valN</code></em>)
<em class="replaceable"><code>col_name</code></em> = <em class="replaceable"><code>val1</code></em> OR ... OR <em class="replaceable"><code>col_name</code></em> = <em class="replaceable"><code>valN</code></em>
</pre><p>
            Each expression is true if
            <em class="replaceable"><code>col_name</code></em> is equal to any of
            several values. These comparisons are equality range
            comparisons (where the <span class="quote">“<span class="quote">range</span>”</span> is a single
            value). The optimizer estimates the cost of reading
            qualifying rows for equality range comparisons as follows:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                If there is a unique index on
                <em class="replaceable"><code>col_name</code></em>, the row estimate
                for each range is 1 because at most one row can have the
                given value.
              </p></li><li class="listitem"><p>
                Otherwise, any index on
                <em class="replaceable"><code>col_name</code></em> is nonunique and the
                optimizer can estimate the row count for each range
                using dives into the index or index statistics.
</p></li></ul>
</div>
<a class="indexterm" name="idm46444353390240"></a><p>
            With index dives, the optimizer makes a dive at each end of
            a range and uses the number of rows in the range as the
            estimate. For example, the expression
            <code class="literal"><em class="replaceable"><code>col_name</code></em> IN (10, 20,
            30)</code> has three equality ranges and the optimizer
            makes two dives per range to generate a row estimate. Each
            pair of dives yields an estimate of the number of rows that
            have the given value.
          </p><p>
            Index dives provide accurate row estimates, but as the
            number of comparison values in the expression increases, the
            optimizer takes longer to generate a row estimate. Use of
            index statistics is less accurate than index dives but
            permits faster row estimation for large value lists.
          </p><p>
            The
            <a class="link" href="server-administration.html#sysvar_eq_range_index_dive_limit"><code class="literal">eq_range_index_dive_limit</code></a>
            system variable enables you to configure the number of
            values at which the optimizer switches from one row
            estimation strategy to the other. To permit use of index
            dives for comparisons of up to <em class="replaceable"><code>N</code></em>
            equality ranges, set
            <a class="link" href="server-administration.html#sysvar_eq_range_index_dive_limit"><code class="literal">eq_range_index_dive_limit</code></a>
            to <em class="replaceable"><code>N</code></em> + 1. To disable use of
            statistics and always use index dives regardless of
            <em class="replaceable"><code>N</code></em>, set
            <a class="link" href="server-administration.html#sysvar_eq_range_index_dive_limit"><code class="literal">eq_range_index_dive_limit</code></a>
            to 0.
          </p><p>
            To update table index statistics for best estimates, use
            <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE TABLE</code></a>.
          </p><p>
            Prior to MySQL 8.0, there is no way of skipping
            the use of index dives to estimate index usefulness, except
            by using the
            <a class="link" href="server-administration.html#sysvar_eq_range_index_dive_limit"><code class="literal">eq_range_index_dive_limit</code></a>
            system variable. In MySQL 8.0, index dive
            skipping is possible for queries that satisfy all these
            conditions:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                The query is for a single table, not a join on multiple
                tables.
              </p></li><li class="listitem"><p>
                A single-index <code class="literal">FORCE INDEX</code> index hint
                is present. The idea is that if index use is forced,
                there is nothing to be gained from the additional
                overhead of performing dives into the index.
              </p></li><li class="listitem"><p>
                The index is nonunique and not a
                <code class="literal">FULLTEXT</code> index.
              </p></li><li class="listitem"><p>
                No subquery is present.
              </p></li><li class="listitem"><p>
                No <code class="literal">DISTINCT</code>, <code class="literal">GROUP
                BY</code>, or <code class="literal">ORDER BY</code> clause is
                present.
</p></li></ul>
</div>
<p>
            For <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN FOR
            CONNECTION</code></a>, the output changes as follows if index
            dives are skipped:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                For traditional output, the <code class="literal">rows</code> and
                <code class="literal">filtered</code> values are
                <code class="literal">NULL</code>.
              </p></li><li class="listitem"><p>
                For JSON output,
                <code class="literal">rows_examined_per_scan</code> and
                <code class="literal">rows_produced_per_join</code> do not appear,
                <code class="literal">skip_index_dive_due_to_force</code> is
                <code class="literal">true</code>, and cost calculations are not
                accurate.
</p></li></ul>
</div>
<p>
            Without <code class="literal">FOR CONNECTION</code>,
            <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output does not
            change when index dives are skipped.
          </p><p>
            After execution of a query for which index dives are
            skipped, the corresponding row in the
            <a class="link" href="information-schema.html#optimizer-trace-table" title="25.18 The INFORMATION_SCHEMA OPTIMIZER_TRACE Table"><code class="literal">INFORMATION_SCHEMA.OPTIMIZER_TRACE</code></a>
            table contains an
            <code class="literal">index_dives_for_range_access</code> value of
            <code class="literal">skipped_due_to_force_index</code>.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h5 class="title"><a name="range-access-skip-scan"></a>Skip Scan Range Access Method</h5>

</div>

</div>

</div>
<p>
            Consider the following scenario:
          </p><pre data-lang="sql" class="programlisting">CREATE TABLE t1 (f1 INT NOT NULL, f2 INT NOT NULL, PRIMARY KEY(f1, f2));
INSERT INTO t1 VALUES
  (1,1), (1,2), (1,3), (1,4), (1,5),
  (2,1), (2,2), (2,3), (2,4), (2,5);
INSERT INTO t1 SELECT f1, f2 + 5 FROM t1;
INSERT INTO t1 SELECT f1, f2 + 10 FROM t1;
INSERT INTO t1 SELECT f1, f2 + 20 FROM t1;
INSERT INTO t1 SELECT f1, f2 + 40 FROM t1;
ANALYZE TABLE t1;

EXPLAIN SELECT f1, f2 FROM t1 WHERE f2 &gt; 40;</pre><p>
            To execute this query, MySQL can choose an index scan to
            fetch all rows (the index includes all columns to be
            selected), then apply the <code class="literal">f2 &gt; 40</code>
            condition from the <code class="literal">WHERE</code> clause to
            produce the final result set.
          </p><p>
            A range scan is more efficient than a full index scan, but
            cannot be used in this case because there is no condition on
            <code class="literal">f1</code>, the first index column. However, as
            of MySQL 8.0.13, the optimizer can perform multiple range
            scans, one for each value of <code class="literal">f1</code>, using a
            method called Skip Scan that is similar to Loose Index Scan
            (see <a class="xref" href="optimization.html#group-by-optimization" title="8.2.1.17 GROUP BY Optimization">Section 8.2.1.17, “GROUP BY Optimization”</a>):
</p>
<div class="orderedlist">
<ol class="orderedlist" type="1"><li class="listitem"><p>
                Skip between distinct values of the first index part,
                <code class="literal">f1</code> (the index prefix).
              </p></li><li class="listitem"><p>
                Perform a subrange scan on each distinct prefix value
                for the <code class="literal">f2 &gt; 40</code> condition on the
                remaining index part.
</p></li></ol>
</div>
<p>
            For the data set shown earlier, the algorithm operates like
            this:
</p>
<div class="orderedlist">
<ol class="orderedlist" type="1"><li class="listitem"><p>
                Get the first distinct value of the first key part
                (<code class="literal">f1 = 1</code>).
              </p></li><li class="listitem"><p>
                Construct the range based on the first and second key
                parts (<code class="literal">f1 = 1 AND f2 &gt; 40</code>).
              </p></li><li class="listitem"><p>
                Perform a range scan.
              </p></li><li class="listitem"><p>
                Get the next distinct value of the first key part
                (<code class="literal">f1 = 2</code>).
              </p></li><li class="listitem"><p>
                Construct the range based on the first and second key
                parts (<code class="literal">f1 = 2 AND f2 &gt; 40</code>).
              </p></li><li class="listitem"><p>
                Perform a range scan.
</p></li></ol>
</div>
<p>
            Using this strategy decreases the number of accessed rows
            because MySQL skips the rows that do not qualify for each
            constructed range. This Skip Scan access method is
            applicable under the following conditions:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                Table T has at least one compound index with key parts
                of the form ([A_1, ..., A_<em class="replaceable"><code>k</code></em>,]
                B_1, ..., B_<em class="replaceable"><code>m</code></em>, C [, D_1, ...,
                D_<em class="replaceable"><code>n</code></em>]). Key parts A and D may
                be empty, but B and C must be nonempty.
              </p></li><li class="listitem"><p>
                The query references only one table.
              </p></li><li class="listitem"><p>
                The query does not use <code class="literal">GROUP BY</code> or
                <code class="literal">DISTINCT</code>.
              </p></li><li class="listitem"><p>
                The query references only columns in the index.
              </p></li><li class="listitem"><p>
                The predicates on A_1, ...,
                A_<em class="replaceable"><code>k</code></em> must be equality
                predicates and they must be constants. This includes the
                <a class="link" href="functions.html#operator_in"><code class="literal">IN()</code></a> operator.
              </p></li><li class="listitem"><p>
                The query must be a conjunctive query; that is, an
                <code class="literal">AND</code> of <code class="literal">OR</code>
                conditions:
                <code class="literal">(<em class="replaceable"><code>cond1</code></em>(<em class="replaceable"><code>key_part1</code></em>)
                OR
                <em class="replaceable"><code>cond2</code></em>(<em class="replaceable"><code>key_part1</code></em>))
                AND
                (<em class="replaceable"><code>cond1</code></em>(<em class="replaceable"><code>key_part2</code></em>)
                OR ...) AND ...</code>
              </p></li><li class="listitem"><p>
                There must be a range condition on C.
              </p></li><li class="listitem"><p>
                Conditions on D columns are permitted. Conditions on D
                must be in conjunction with the range condition on C.
</p></li></ul>
</div>
<p>
            Use of Skip Scan is indicated in <code class="literal">EXPLAIN</code>
            output as follows:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                <code class="literal">Using index for skip scan</code> in the
                <code class="literal">Extra</code> column indicates that the loose
                index Skip Scan access method is used.
              </p></li><li class="listitem"><p>
                If the index can be used for Skip Scan, the index should
                be visible in the <code class="literal">possible_keys</code>
                column.
</p></li></ul>
</div>
<p>
            Use of Skip Scan is indicated in optimizer trace output by a
            <code class="literal">"skip scan"</code> element of this form:
          </p><pre data-lang="json" class="programlisting">"skip_scan_range": {
  "type": "skip_scan",
  "index": <em class="replaceable"><code>index_used_for_skip_scan</code></em>,
  "key_parts_used_for_access": [<em class="replaceable"><code>key_parts_used_for_access</code></em>],
  "range": [<em class="replaceable"><code>range</code></em>]
}
</pre><p>
            You may also see a
            <code class="literal">"best_skip_scan_summary"</code> element. If Skip
            Scan is chosen as the best range access variant, a
            <code class="literal">"chosen_range_access_summary"</code> is written.
            If Skip Scan is chosen as the overall best access method, a
            <code class="literal">"best_access_path"</code> element is present.
          </p><p>
            Use of Skip Scan is subject to the value of the
            <code class="literal">skip_scan</code> flag of the
            <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
            variable. See <a class="xref" href="optimization.html#switchable-optimizations" title="8.9.2 Switchable Optimizations">Section 8.9.2, “Switchable Optimizations”</a>. By
            default, this flag is <code class="literal">on</code>. To disable it,
            set <code class="literal">skip_scan</code> to <code class="literal">off</code>.
          </p><p>
            In addition to using the
            <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
            variable to control optimizer use of Skip Scan session-wide,
            MySQL supports optimizer hints to influence the optimizer on
            a per-statement basis. See
            <a class="xref" href="optimization.html#optimizer-hints" title="8.9.3 Optimizer Hints">Section 8.9.3, “Optimizer Hints”</a>.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h5 class="title"><a name="row-constructor-range-optimization"></a>Range Optimization of Row Constructor Expressions</h5>

</div>

</div>

</div>
<p>
            The optimizer is able to apply the range scan access method
            to queries of this form:
          </p><pre data-lang="sql" class="programlisting">SELECT ... FROM t1 WHERE ( col_1, col_2 ) IN (( 'a', 'b' ), ( 'c', 'd' ));</pre><p>
            Previously, for range scans to be used, it was necessary to
            write the query as:
          </p><pre data-lang="sql" class="programlisting">SELECT ... FROM t1 WHERE ( col_1 = 'a' AND col_2 = 'b' )
OR ( col_1 = 'c' AND col_2 = 'd' );</pre><p>
            For the optimizer to use a range scan, queries must satisfy
            these conditions:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                Only <a class="link" href="functions.html#operator_in"><code class="literal">IN()</code></a> predicates are
                used, not <a class="link" href="functions.html#operator_not-in"><code class="literal">NOT IN()</code></a>.
              </p></li><li class="listitem"><p>
                On the left side of the
                <a class="link" href="functions.html#operator_in"><code class="literal">IN()</code></a> predicate, the row
                constructor contains only column references.
              </p></li><li class="listitem"><p>
                On the right side of the
                <a class="link" href="functions.html#operator_in"><code class="literal">IN()</code></a> predicate, row
                constructors contain only runtime constants, which are
                either literals or local column references that are
                bound to constants during execution.
              </p></li><li class="listitem"><p>
                On the right side of the
                <a class="link" href="functions.html#operator_in"><code class="literal">IN()</code></a> predicate, there is
                more than one row constructor.
</p></li></ul>
</div>
<p>
            For more information about the optimizer and row
            constructors, see
            <a class="xref" href="optimization.html#row-constructor-optimization" title="8.2.1.22 Row Constructor Expression Optimization">Section 8.2.1.22, “Row Constructor Expression Optimization”</a>
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h5 class="title"><a name="range-optimization-memory-use"></a>Limiting Memory Use for Range Optimization</h5>

</div>

</div>

</div>
<p>
            To control the memory available to the range optimizer, use
            the
            <a class="link" href="server-administration.html#sysvar_range_optimizer_max_mem_size"><code class="literal">range_optimizer_max_mem_size</code></a>
            system variable:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                A value of 0 means <span class="quote">“<span class="quote">no limit.</span>”</span>
              </p></li><li class="listitem"><p>
                With a value greater than 0, the optimizer tracks the
                memory consumed when considering the range access
                method. If the specified limit is about to be exceeded,
                the range access method is abandoned and other methods,
                including a full table scan, are considered instead.
                This could be less optimal. If this happens, the
                following warning occurs (where
                <em class="replaceable"><code>N</code></em> is the current
                <a class="link" href="server-administration.html#sysvar_range_optimizer_max_mem_size"><code class="literal">range_optimizer_max_mem_size</code></a>
                value):
              </p><pre data-lang="none" class="programlisting">Warning    3170    Memory capacity of <em class="replaceable"><code>N</code></em> bytes for
                   'range_optimizer_max_mem_size' exceeded. Range
                   optimization was not done for this query.
</pre></li><li class="listitem"><p>
                For <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a> and
                <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a> statements, if the
                optimizer falls back to a full table scan and the
                <a class="link" href="server-administration.html#sysvar_sql_safe_updates"><code class="literal">sql_safe_updates</code></a> system
                variable is enabled, an error occurs rather than a
                warning because, in effect, no key is used to determine
                which rows to modify. For more information, see
                <a class="xref" href="programs.html#safe-updates" title="Using Safe-Updates Mode (--safe-updates)">Using Safe-Updates Mode (--safe-updates)</a>.
</p></li></ul>
</div>
<p>
            For individual queries that exceed the available range
            optimization memory and for which the optimizer falls back
            to less optimal plans, increasing the
            <a class="link" href="server-administration.html#sysvar_range_optimizer_max_mem_size"><code class="literal">range_optimizer_max_mem_size</code></a>
            value may improve performance.
          </p><p>
            To estimate the amount of memory needed to process a range
            expression, use these guidelines:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                For a simple query such as the following, where there is
                one candidate key for the range access method, each
                predicate combined with <a class="link" href="functions.html#operator_or"><code class="literal">OR</code></a>
                uses approximately 230 bytes:
              </p><pre data-lang="sql" class="programlisting">SELECT COUNT(*) FROM t
WHERE a=1 OR a=2 OR a=3 OR .. . a=<em class="replaceable"><code>N</code></em>;
</pre></li><li class="listitem"><p>
                Similarly for a query such as the following, each
                predicate combined with <a class="link" href="functions.html#operator_and"><code class="literal">AND</code></a>
                uses approximately 125 bytes:
              </p><pre data-lang="sql" class="programlisting">SELECT COUNT(*) FROM t
WHERE a=1 AND b=1 AND c=1 ... <em class="replaceable"><code>N</code></em>;
</pre></li><li class="listitem"><p>
                For a query with <a class="link" href="functions.html#operator_in"><code class="literal">IN()</code></a>
                predicates:
              </p><pre data-lang="sql" class="programlisting">SELECT COUNT(*) FROM t
WHERE a IN (1,2, ..., <em class="replaceable"><code>M</code></em>) AND b IN (1,2, ..., <em class="replaceable"><code>N</code></em>);
</pre><p>
                Each literal value in an
                <a class="link" href="functions.html#operator_in"><code class="literal">IN()</code></a> list counts as a
                predicate combined with <a class="link" href="functions.html#operator_or"><code class="literal">OR</code></a>.
                If there are two <a class="link" href="functions.html#operator_in"><code class="literal">IN()</code></a>
                lists, the number of predicates combined with
                <a class="link" href="functions.html#operator_or"><code class="literal">OR</code></a> is the product of the
                number of literal values in each list. Thus, the number
                of predicates combined with
                <a class="link" href="functions.html#operator_or"><code class="literal">OR</code></a> in the preceding case is
                <em class="replaceable"><code>M</code></em> ×
                <em class="replaceable"><code>N</code></em>.
</p></li></ul>
</div>

</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="index-merge-optimization"></a>8.2.1.3 Index Merge Optimization</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444353240000"></a><a class="indexterm" name="idm46444353238960"></a><a class="indexterm" name="idm46444353237872"></a><p>
          The <span class="firstterm">Index Merge</span> access
          method retrieves rows with multiple
          <a class="link" href="optimization.html#jointype_range"><code class="literal">range</code></a> scans and merges
          their results into one. This access method merges index scans
          from a single table only, not scans across multiple tables.
          The merge can produce unions, intersections, or
          unions-of-intersections of its underlying scans.
        </p><p>
          Example queries for which Index Merge may be used:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM <em class="replaceable"><code>tbl_name</code></em> WHERE <em class="replaceable"><code>key1</code></em> = 10 OR <em class="replaceable"><code>key2</code></em> = 20;

SELECT * FROM <em class="replaceable"><code>tbl_name</code></em>
  WHERE (<em class="replaceable"><code>key1</code></em> = 10 OR <em class="replaceable"><code>key2</code></em> = 20) AND <em class="replaceable"><code>non_key</code></em> = 30;

SELECT * FROM t1, t2
  WHERE (t1.<em class="replaceable"><code>key1</code></em> IN (1,2) OR t1.<em class="replaceable"><code>key2</code></em> LIKE '<em class="replaceable"><code>value</code></em>%')
  AND t2.<em class="replaceable"><code>key1</code></em> = t1.<em class="replaceable"><code>some_col</code></em>;

SELECT * FROM t1, t2
  WHERE t1.<em class="replaceable"><code>key1</code></em> = 1
  AND (t2.<em class="replaceable"><code>key1</code></em> = t1.<em class="replaceable"><code>some_col</code></em> OR t2.<em class="replaceable"><code>key2</code></em> = t1.<em class="replaceable"><code>some_col2</code></em>);
</pre>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
            The Index Merge optimization algorithm has the following
            known limitations:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                If your query has a complex <code class="literal">WHERE</code>
                clause with deep
                <a class="link" href="functions.html#operator_and"><code class="literal">AND</code></a>/<a class="link" href="functions.html#operator_or"><code class="literal">OR</code></a>
                nesting and MySQL does not choose the optimal plan, try
                distributing terms using the following identity
                transformations:
              </p><pre data-lang="sql" class="programlisting">(<em class="replaceable"><code>x</code></em> AND <em class="replaceable"><code>y</code></em>) OR <em class="replaceable"><code>z</code></em> =&gt; (<em class="replaceable"><code>x</code></em> OR <em class="replaceable"><code>z</code></em>) AND (<em class="replaceable"><code>y</code></em> OR <em class="replaceable"><code>z</code></em>)
(<em class="replaceable"><code>x</code></em> OR <em class="replaceable"><code>y</code></em>) AND <em class="replaceable"><code>z</code></em> =&gt; (<em class="replaceable"><code>x</code></em> AND <em class="replaceable"><code>z</code></em>) OR (<em class="replaceable"><code>y</code></em> AND <em class="replaceable"><code>z</code></em>)
</pre></li><li class="listitem"><p>
                Index Merge is not applicable to full-text indexes.
</p></li></ul>
</div>

</div>
<p>
          In <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output, the Index
          Merge method appears as
          <a class="link" href="optimization.html#jointype_index_merge"><code class="literal">index_merge</code></a> in the
          <code class="literal">type</code> column. In this case, the
          <code class="literal">key</code> column contains a list of indexes used,
          and <code class="literal">key_len</code> contains a list of the longest
          key parts for those indexes.
        </p><p>
          The Index Merge access method has several algorithms, which
          are displayed in the <code class="literal">Extra</code> field of
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <code class="literal">Using intersect(...)</code>
            </p></li><li class="listitem"><p>
              <code class="literal">Using union(...)</code>
            </p></li><li class="listitem"><p>
              <code class="literal">Using sort_union(...)</code>
</p></li></ul>
</div>
<p>
          The following sections describe these algorithms in greater
          detail. The optimizer chooses between different possible Index
          Merge algorithms and other access methods based on cost
          estimates of the various available options.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a class="xref" href="optimization.html#index-merge-intersection" title="Index Merge Intersection Access Algorithm">Index Merge Intersection Access Algorithm</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#index-merge-union" title="Index Merge Union Access Algorithm">Index Merge Union Access Algorithm</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#index-merge-sort-union" title="Index Merge Sort-Union Access Algorithm">Index Merge Sort-Union Access Algorithm</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#index-merge-influencing" title="Influencing Index Merge Optimization">Influencing Index Merge Optimization</a></p></li></ul>
</div>

<div class="simplesect">

<div class="titlepage">
<div>

<div class="simple">
<h5 class="title"><a name="index-merge-intersection"></a>Index Merge Intersection Access Algorithm</h5>

</div>

</div>

</div>
<p>
            This access algorithm is applicable when a
            <code class="literal">WHERE</code> clause is converted to several
            range conditions on different keys combined with
            <a class="link" href="functions.html#operator_and"><code class="literal">AND</code></a>, and each condition is one
            of the following:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                An <em class="replaceable"><code>N</code></em>-part expression of this
                form, where the index has exactly
                <em class="replaceable"><code>N</code></em> parts (that is, all index
                parts are covered):
              </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>key_part1</code></em> = <em class="replaceable"><code>const1</code></em> AND <em class="replaceable"><code>key_part2</code></em> = <em class="replaceable"><code>const2</code></em> ... AND <em class="replaceable"><code>key_partN</code></em> = <em class="replaceable"><code>constN</code></em>
</pre></li><li class="listitem"><p>
                Any range condition over the primary key of an
                <code class="literal">InnoDB</code> table.
</p></li></ul>
</div>
<p>
            Examples:
          </p><pre data-lang="sql" class="programlisting">SELECT * FROM <em class="replaceable"><code>innodb_table</code></em>
  WHERE <em class="replaceable"><code>primary_key</code></em> &lt; 10 AND <em class="replaceable"><code>key_col1</code></em> = 20;

SELECT * FROM <em class="replaceable"><code>tbl_name</code></em>
  WHERE <em class="replaceable"><code>key1_part1</code></em> = 1 AND <em class="replaceable"><code>key1_part2</code></em> = 2 AND <em class="replaceable"><code>key2</code></em> = 2;
</pre><p>
            The Index Merge intersection algorithm performs simultaneous
            scans on all used indexes and produces the intersection of
            row sequences that it receives from the merged index scans.
          </p><p>
            If all columns used in the query are covered by the used
            indexes, full table rows are not retrieved
            (<a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output contains
            <code class="literal">Using index</code> in <code class="literal">Extra</code>
            field in this case). Here is an example of such a query:
          </p><pre data-lang="sql" class="programlisting">SELECT COUNT(*) FROM t1 WHERE key1 = 1 AND key2 = 1;</pre><p>
            If the used indexes do not cover all columns used in the
            query, full rows are retrieved only when the range
            conditions for all used keys are satisfied.
          </p><p>
            If one of the merged conditions is a condition over the
            primary key of an <code class="literal">InnoDB</code> table, it is not
            used for row retrieval, but is used to filter out rows
            retrieved using other conditions.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h5 class="title"><a name="index-merge-union"></a>Index Merge Union Access Algorithm</h5>

</div>

</div>

</div>
<p>
            The criteria for this algorithm are similar to those for the
            Index Merge intersection algorithm. The algorithm is
            applicable when the table's <code class="literal">WHERE</code>
            clause is converted to several range conditions on different
            keys combined with <a class="link" href="functions.html#operator_or"><code class="literal">OR</code></a>, and each
            condition is one of the following:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                An <em class="replaceable"><code>N</code></em>-part expression of this
                form, where the index has exactly
                <em class="replaceable"><code>N</code></em> parts (that is, all index
                parts are covered):
              </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>key_part1</code></em> = <em class="replaceable"><code>const1</code></em> AND <em class="replaceable"><code>key_part2</code></em> = <em class="replaceable"><code>const2</code></em> ... AND <em class="replaceable"><code>key_partN</code></em> = <em class="replaceable"><code>constN</code></em>
</pre></li><li class="listitem"><p>
                Any range condition over a primary key of an
                <code class="literal">InnoDB</code> table.
              </p></li><li class="listitem"><p>
                A condition for which the Index Merge intersection
                algorithm is applicable.
</p></li></ul>
</div>
<p>
            Examples:
          </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1
  WHERE <em class="replaceable"><code>key1</code></em> = 1 OR <em class="replaceable"><code>key2</code></em> = 2 OR <em class="replaceable"><code>key3</code></em> = 3;

SELECT * FROM <em class="replaceable"><code>innodb_table</code></em>
  WHERE (<em class="replaceable"><code>key1</code></em> = 1 AND <em class="replaceable"><code>key2</code></em> = 2)
     OR (<em class="replaceable"><code>key3</code></em> = 'foo' AND <em class="replaceable"><code>key4</code></em> = 'bar') AND <em class="replaceable"><code>key5</code></em> = 5;
</pre>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h5 class="title"><a name="index-merge-sort-union"></a>Index Merge Sort-Union Access Algorithm</h5>

</div>

</div>

</div>
<p>
            This access algorithm is applicable when the
            <code class="literal">WHERE</code> clause is converted to several
            range conditions combined by
            <a class="link" href="functions.html#operator_or"><code class="literal">OR</code></a>, but the Index Merge union
            algorithm is not applicable.
          </p><p>
            Examples:
          </p><pre data-lang="sql" class="programlisting">SELECT * FROM <em class="replaceable"><code>tbl_name</code></em>
  WHERE <em class="replaceable"><code>key_col1</code></em> &lt; 10 OR <em class="replaceable"><code>key_col2</code></em> &lt; 20;

SELECT * FROM <em class="replaceable"><code>tbl_name</code></em>
  WHERE (<em class="replaceable"><code>key_col1</code></em> &gt; 10 OR <em class="replaceable"><code>key_col2</code></em> = 20) AND <em class="replaceable"><code>nonkey_col</code></em> = 30;
</pre><p>
            The difference between the sort-union algorithm and the
            union algorithm is that the sort-union algorithm must first
            fetch row IDs for all rows and sort them before returning
            any rows.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h5 class="title"><a name="index-merge-influencing"></a>Influencing Index Merge Optimization</h5>

</div>

</div>

</div>
<p>
            Use of Index Merge is subject to the value of the
            <code class="literal">index_merge</code>,
            <code class="literal">index_merge_intersection</code>,
            <code class="literal">index_merge_union</code>, and
            <code class="literal">index_merge_sort_union</code> flags of the
            <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
            variable. See <a class="xref" href="optimization.html#switchable-optimizations" title="8.9.2 Switchable Optimizations">Section 8.9.2, “Switchable Optimizations”</a>. By
            default, all those flags are <code class="literal">on</code>. To
            enable only certain algorithms, set
            <code class="literal">index_merge</code> to <code class="literal">off</code>,
            and enable only such of the others as should be permitted.
          </p><p>
            In addition to using the
            <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
            variable to control optimizer use of the Index Merge
            algorithms session-wide, MySQL supports optimizer hints to
            influence the optimizer on a per-statement basis. See
            <a class="xref" href="optimization.html#optimizer-hints" title="8.9.3 Optimizer Hints">Section 8.9.3, “Optimizer Hints”</a>.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="hash-joins"></a>8.2.1.4 Hash Join Optimization</h4>

</div>

</div>

</div>
<p>
          Beginning with MySQL 8.0.18, MySQL employs a hash join for any
          query for which each join has an equi-join condition and uses
          no indexes, such as this one:
        </p><pre data-lang="sql" class="programlisting">SELECT *
    FROM t1
    JOIN t2
        ON t1.c1=t2.c1;</pre><p>
          A hash join is usually faster than and is intended to be used
          in such cases instead of the block nested loop algorithm (see
          <a class="xref" href="optimization.html#block-nested-loop-join-algorithm" title="Block Nested-Loop Join Algorithm">Block Nested-Loop Join Algorithm</a>) employed
          in previous versions of MySQL.
        </p><p>
          In the example just shown and the remaining examples in this
          section, we assume that the three tables
          <code class="literal">t1</code>, <code class="literal">t2</code>, and
          <code class="literal">t3</code> have been created using the following
          statements:
        </p><pre data-lang="sql" class="programlisting">CREATE TABLE t1 (c1 INT, c2 INT);
CREATE TABLE t2 (c1 INT, c2 INT);
CREATE TABLE t3 (c1 INT, c2 INT);</pre><p>
          You can see that a hash join is being employed by using
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN
          FORMAT=TREE</code></a>, like this:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN FORMAT=TREE</code></strong>
    -&gt; <strong class="userinput"><code>SELECT *</code></strong>
    -&gt;     <strong class="userinput"><code>FROM t1</code></strong>
    -&gt;     <strong class="userinput"><code>JOIN t2</code></strong>
    -&gt;         <strong class="userinput"><code>ON t1.c1=t2.c1\G</code></strong>
*************************** 1. row ***************************
EXPLAIN: -&gt; Inner hash join (t2.c1 = t1.c1)  (cost=0.70 rows=1)
    -&gt; Table scan on t2  (cost=0.35 rows=1)
    -&gt; Hash
        -&gt; Table scan on t1  (cost=0.35 rows=1)
</pre><p>
          To see whether hash joins are being used for a given join, you
          must use <code class="literal">EXPLAIN</code> with the
          <code class="literal">FORMAT=TREE</code> option.
          <a class="link" href="sql-statements.html#explain-analyze" title="Obtaining Information with EXPLAIN ANALYZE"><code class="literal">EXPLAIN ANALYZE</code></a> also displays
          information about hash joins used.
        </p><p>
          The hash join is used for queries involving multiple joins as
          well, as long as at least one join condition for each pair of
          tables is an equi-join, like the query shown here:
        </p><pre data-lang="sql" class="programlisting">SELECT *
    FROM t1
    JOIN t2
        ON (t1.c1 = t2.c1 AND t1.c2 &lt; t2.c2)
    JOIN t3
        ON (t2.c1 = t3.c1);</pre><p>
          In cases like the one just shown, any extra conditions which
          are not equi-joins are applied as filters after the join is
          executed. This can be seen here in the output of
          <code class="literal">EXPLAIN FORMAT=TREE</code>:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN FORMAT=TREE</code></strong>
    -&gt; <strong class="userinput"><code>SELECT *</code></strong>
    -&gt;     <strong class="userinput"><code>FROM t1</code></strong>
    -&gt;     <strong class="userinput"><code>JOIN t2</code></strong>
    -&gt;         <strong class="userinput"><code>ON (t1.c1 = t2.c1 AND t1.c2 &lt; t2.c2)</code></strong>
    -&gt;     <strong class="userinput"><code>JOIN t3</code></strong>
    -&gt;         <strong class="userinput"><code>ON (t2.c1 = t3.c1)\G</code></strong>
*************************** 1. row ***************************
EXPLAIN: -&gt; Inner hash join (t3.c1 = t1.c1)  (cost=1.05 rows=1)
    -&gt; Table scan on t3  (cost=0.35 rows=1)
    -&gt; Hash
        -&gt; Filter: (t1.c2 &lt; t2.c2)  (cost=0.70 rows=1)
            -&gt; Inner hash join (t2.c1 = t1.c1)  (cost=0.70 rows=1)
                -&gt; Table scan on t2  (cost=0.35 rows=1)
                -&gt; Hash
                    -&gt; Table scan on t1  (cost=0.35 rows=1)
</pre><p>
          As also can be seen from the output just shown, multiple hash
          joins can be (and are) used for joins having multiple
          equi-join conditions.
        </p><p>
          A hash join cannot be used if any pair of joined tables does
          not have at least one equi-join condition, as can be seen
          here:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN FORMAT=TREE</code></strong>
    -&gt;     <strong class="userinput"><code>SELECT *</code></strong>
    -&gt;         <strong class="userinput"><code>FROM t1</code></strong>
    -&gt;         <strong class="userinput"><code>JOIN t2</code></strong>
    -&gt;             <strong class="userinput"><code>ON (t1.c1 = t2.c1)</code></strong>
    -&gt;         <strong class="userinput"><code>JOIN t3</code></strong>
    -&gt;             <strong class="userinput"><code>ON (t2.c1 &lt; t3.c1)\G</code></strong>
*************************** 1. row ***************************
EXPLAIN: &lt;not executable by iterator executor&gt;
</pre><p>
          In this case, the slower block nested loop algorithm is
          employed, as in versions previous to MySQL 8.0.18 where no
          indexes are available:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN</code></strong>
    -&gt;     <strong class="userinput"><code>SELECT *</code></strong>
    -&gt;         <strong class="userinput"><code>FROM t1</code></strong>
    -&gt;         <strong class="userinput"><code>JOIN t2</code></strong>
    -&gt;             <strong class="userinput"><code>ON (t1.c1 = t2.c1)</code></strong>
    -&gt;         <strong class="userinput"><code>JOIN t3</code></strong>
    -&gt;             <strong class="userinput"><code>ON (t2.c1 &lt; t3.c1)\G</code></strong>
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: t1
   partitions: NULL
         type: ALL
possible_keys: NULL
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 1
     filtered: 100.00
        Extra: NULL
*************************** 2. row ***************************
           id: 1
  select_type: SIMPLE
        table: t2
   partitions: NULL
         type: ALL
possible_keys: NULL
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 1
     filtered: 100.00
        Extra: Using where; Using join buffer (Block Nested Loop)
*************************** 3. row ***************************
           id: 1
  select_type: SIMPLE
        table: t3
   partitions: NULL
         type: ALL
possible_keys: NULL
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 1
     filtered: 100.00
        Extra: Using where; Using join buffer (Block Nested Loop)
</pre><p>
          A hash join is also applied for a Cartesian product—that
          is, when no join condition is specified, as shown here:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN FORMAT=TREE</code></strong>
    -&gt; <strong class="userinput"><code>SELECT *</code></strong>
    -&gt;     <strong class="userinput"><code>FROM t1</code></strong>
    -&gt;     <strong class="userinput"><code>JOIN t2</code></strong>
    -&gt;     <strong class="userinput"><code>WHERE t1.c2 &gt; 50\G</code></strong>
*************************** 1. row ***************************
EXPLAIN: -&gt; Inner hash join  (cost=0.70 rows=1)
    -&gt; Table scan on t2  (cost=0.35 rows=1)
    -&gt; Hash
        -&gt; Filter: (t1.c2 &gt; 50)  (cost=0.35 rows=1)
            -&gt; Table scan on t1  (cost=0.35 rows=1)
</pre><p>
          In MySQL 8.0.20 and later, it is no longer necessary for the
          join to contain at least one equi-join condition in order for
          a hash join to be used. This means that the types of queries
          can be optimized using hash joins include those in the
          following list (with examples):
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <span class="emphasis"><em>Inner non-equi-join</em></span>:
            </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN FORMAT=TREE SELECT * FROM t1 JOIN t2 ON t1.c1 &lt; t2.c1\G</code></strong>
*************************** 1. row ***************************
EXPLAIN: -&gt; Filter: (t1.c1 &lt; t2.c1)  (cost=4.70 rows=12)
    -&gt; Inner hash join (no condition)  (cost=4.70 rows=12)
        -&gt; Table scan on t2  (cost=0.08 rows=6)
        -&gt; Hash
            -&gt; Table scan on t1  (cost=0.85 rows=6)
</pre></li><li class="listitem"><p>
              <span class="emphasis"><em>Semijoin</em></span>:
            </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN FORMAT=TREE SELECT * FROM t1 </code></strong>
    -&gt;     <strong class="userinput"><code>WHERE t1.c1 IN (SELECT t2.c2 FROM t2)\G</code></strong>
*************************** 1. row ***************************
EXPLAIN: -&gt; Nested loop inner join
    -&gt; Filter: (t1.c1 is not null)  (cost=0.85 rows=6)
        -&gt; Table scan on t1  (cost=0.85 rows=6)
    -&gt; Single-row index lookup on &lt;subquery2&gt; using &lt;auto_distinct_key&gt; (c2=t1.c1)
        -&gt; Materialize with deduplication
            -&gt; Filter: (t2.c2 is not null)  (cost=0.85 rows=6)
                -&gt; Table scan on t2  (cost=0.85 rows=6)
</pre></li><li class="listitem"><p>
              <span class="emphasis"><em>Antijoin</em></span>:
            </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN FORMAT=TREE SELECT * FROM t2 </code></strong>
    -&gt;     <strong class="userinput"><code>WHERE NOT EXISTS (SELECT * FROM t1 WHERE t1.col1 = t2.col1)\G</code></strong>
*************************** 1. row ***************************
EXPLAIN: -&gt; Nested loop antijoin
    -&gt; Table scan on t2  (cost=0.85 rows=6)
    -&gt; Single-row index lookup on &lt;subquery2&gt; using &lt;auto_distinct_key&gt; (c1=t2.c1)
        -&gt; Materialize with deduplication
            -&gt; Filter: (t1.c1 is not null)  (cost=0.85 rows=6)
                -&gt; Table scan on t1  (cost=0.85 rows=6)
</pre></li><li class="listitem"><p>
              <span class="emphasis"><em>Left outer join</em></span>:
            </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN FORMAT=TREE SELECT * FROM t1 LEFT JOIN t2 ON t1.c1 = t2.c1\G</code></strong>
*************************** 1. row ***************************
EXPLAIN: -&gt; Left hash join (t2.c1 = t1.c1)  (cost=3.99 rows=36)
    -&gt; Table scan on t1  (cost=0.85 rows=6)
    -&gt; Hash
        -&gt; Table scan on t2  (cost=0.14 rows=6)
</pre></li><li class="listitem"><p>
              <span class="emphasis"><em>Right outer join</em></span> (observe that MySQL
              rewrites all right outer joins as left outer joins):
            </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN FORMAT=TREE SELECT * FROM t1 RIGHT JOIN t2 ON t1.c1 = t2.c1\G</code></strong>
*************************** 1. row ***************************
EXPLAIN: -&gt; Left hash join (t1.c1 = t2.c1)  (cost=3.99 rows=36)
    -&gt; Table scan on t2  (cost=0.85 rows=6)
    -&gt; Hash
        -&gt; Table scan on t1  (cost=0.14 rows=6)
</pre></li></ul>
</div>
<p>
          By default, MySQL employs hash joins whenever possible. In
          MySQL 8.0.18 only, it is possible to control whether hash
          joins are used in either of two ways:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              On the global or session level by using
              <code class="literal">hash_join=on</code> or
              <code class="literal">hash_join=off</code> as part of the setting
              for the <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a>
              server system variable. The default is
              <code class="literal">hash_join=on</code>.
            </p></li><li class="listitem"><p>
              On a case-by-case basis, by using one of the optimizer
              hints <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">HASH_JOIN</code></a> or
              <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">NO_HASH_JOIN</code></a> as part of
              a given join.
</p></li></ul>
</div>
<p>
          (Beginning with MySQL 8.0.19, the <code class="literal">hash_join</code>
          optimizer switch and the
          <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">HASH_JOIN</code></a> and
          <code class="literal">NO_HASH_JOIN</code> optimizer hints no longer have
          any effect.)
        </p><p>
          Memory usage by hash joins can be controlled using the
          <a class="link" href="server-administration.html#sysvar_join_buffer_size"><code class="literal">join_buffer_size</code></a> system
          variable; a hash join cannot use more memory than this amount.
          When the memory required for a hash join exceeds the amount
          available, MySQL handles this by using files on disk. If this
          happens, you should be aware that the join may not succeed if
          a hash join cannot fit into memory and it creates more files
          than set for
          <a class="link" href="server-administration.html#sysvar_open_files_limit"><code class="literal">open_files_limit</code></a>. To avoid
          such problems, make either of the following changes:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Increase <code class="literal">join_buffer_size</code> so that the
              hash join does not spill over to disk.
            </p></li><li class="listitem"><p>
              Increase <code class="literal">open_files_limit</code>.
</p></li></ul>
</div>
<p>
          Beginning with MySQL 8.0.18, join buffers for hash joins using
          Batched Key Access (BKA) are allocated incrementally; thus,
          for inner-join BKA, which uses incremental allocation, you can
          set <a class="link" href="server-administration.html#sysvar_join_buffer_size"><code class="literal">join_buffer_size</code></a> higher
          without small queries allocating very large amounts of RAM,
          but outer joins allocate the entire buffer. In MySQL 8.0.20
          and later, hash joins are used for outer joins (including
          antijoins and semijoins) as well, so this is no longer an
          issue.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="condition-pushdown-optimization"></a>8.2.1.5 Engine Condition Pushdown Optimization</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444353043248"></a><a class="indexterm" name="idm46444353042160"></a><p>
          This optimization improves the efficiency of direct
          comparisons between a nonindexed column and a constant. In
          such cases, the condition is <span class="quote">“<span class="quote">pushed down</span>”</span> to the
          storage engine for evaluation. This optimization can be used
          only by the <a class="link" href="mysql-cluster.html" title="Chapter 22 MySQL NDB Cluster 8.0"><code class="literal">NDB</code></a> storage engine.
        </p><p>
          For NDB Cluster, this optimization can eliminate the need to
          send nonmatching rows over the network between the
          cluster's data nodes and the MySQL server that issued the
          query, and can speed up queries where it is used by a factor
          of 5 to 10 times over cases where condition pushdown could be
          but is not used.
        </p><p>
          Suppose that an NDB Cluster table is defined as follows:
        </p><pre data-lang="sql" class="programlisting">CREATE TABLE t1 (
    a INT,
    b INT,
    KEY(a)
) ENGINE=NDB;</pre><p>
          Condition pushdown can be used with queries such as the one
          shown here, which includes a comparison between a nonindexed
          column and a constant:
        </p><pre data-lang="sql" class="programlisting">SELECT a, b FROM t1 WHERE b = 10;</pre><p>
          The use of condition pushdown can be seen in the output of
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN SELECT a,b FROM t1 WHERE b = 10\G</code></strong>
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: t1
         type: ALL
possible_keys: NULL
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 10
        Extra: Using where with pushed condition
</pre><p>
          However, condition pushdown <span class="emphasis"><em>cannot</em></span> be
          used with the following query:
        </p><pre data-lang="sql" class="programlisting">SELECT a,b FROM t1 WHERE a = 10;</pre><p>
          Condition pushdown is not applicable here because an index
          exists on column <code class="literal">a</code>. (An index access method
          would be more efficient and so would be chosen in preference
          to condition pushdown.)
        </p><p>
          Condition pushdown may also be employed when an indexed column
          is compared with a constant using a <code class="literal">&gt;</code> or
          <code class="literal">&lt;</code> operator:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN SELECT a, b FROM t1 WHERE a &lt; 2\G</code></strong>
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: t1
         type: range
possible_keys: a
          key: a
      key_len: 5
          ref: NULL
         rows: 2
        Extra: Using where with pushed condition
</pre><p>
          Other supported comparisons for condition pushdown include the
          following:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <code class="literal"><em class="replaceable"><code>column</code></em> [NOT] LIKE
              <em class="replaceable"><code>pattern</code></em></code>
            </p><p>
              <em class="replaceable"><code>pattern</code></em> must be a string
              literal containing the pattern to be matched; for syntax,
              see <a class="xref" href="functions.html#string-comparison-functions" title="12.7.1 String Comparison Functions and Operators">Section 12.7.1, “String Comparison Functions and Operators”</a>.
            </p></li><li class="listitem"><p>
              <code class="literal"><em class="replaceable"><code>column</code></em> IS [NOT]
              NULL</code>
            </p></li><li class="listitem"><p>
              <code class="literal"><em class="replaceable"><code>column</code></em> IN
              (<em class="replaceable"><code>value_list</code></em>)</code>
            </p><p>
              Each item in the <em class="replaceable"><code>value_list</code></em>
              must be a constant, literal value.
            </p></li><li class="listitem"><p>
              <code class="literal"><em class="replaceable"><code>column</code></em> BETWEEN
              <em class="replaceable"><code>constant1</code></em> AND
              <em class="replaceable"><code>constant2</code></em></code>
            </p><p>
              <em class="replaceable"><code>constant1</code></em> and
              <em class="replaceable"><code>constant2</code></em> must each be a
              constant, literal value.
</p></li></ul>
</div>
<p>
          In all of the cases in the preceding list, it is possible for
          the condition to be converted into the form of one or more
          direct comparisons between a column and a constant.
        </p><p>
          Engine condition pushdown is enabled by default. To disable it
          at server startup, set the
          <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
          variable. For example, in a <code class="filename">my.cnf</code> file,
          use these lines:
        </p><pre data-lang="ini" class="programlisting">[mysqld]
optimizer_switch=engine_condition_pushdown=off</pre><p>
          At runtime, disable condition pushdown like this:
        </p><pre data-lang="sql" class="programlisting">SET optimizer_switch='engine_condition_pushdown=off';</pre><p><b>Limitations. </b>
            Engine condition pushdown is subject to the following
            limitations:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Condition pushdown is supported only by the
              <a class="link" href="mysql-cluster.html" title="Chapter 22 MySQL NDB Cluster 8.0"><code class="literal">NDB</code></a> storage engine.
            </p></li><li class="listitem"><p>
              Prior to NDB 8.0.18, columns could be compared with
              constants or expressions which evaluate to constant values
              only. In NDB 8.0.18 and later, columns can be compared
              with one another as long as they are of exactly the same
              type, including the same signedness, length, character
              set, precision, and scale, where these are applicable.
            </p></li><li class="listitem"><p>
              Columns used in comparisons cannot be of any of the
              <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a> or
              <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">TEXT</code></a> types. This exclusion
              extends to <a class="link" href="data-types.html#json" title="11.5 The JSON Data Type"><code class="literal">JSON</code></a>,
              <a class="link" href="data-types.html#bit-type" title="11.1.5 Bit-Value Type - BIT"><code class="literal">BIT</code></a>, and
              <a class="link" href="data-types.html#enum" title="11.3.5 The ENUM Type"><code class="literal">ENUM</code></a> columns as well.
            </p></li><li class="listitem"><p>
              A string value to be compared with a column must use the
              same collation as the column.
            </p></li><li class="listitem"><p>
              Joins are not directly supported; conditions involving
              multiple tables are pushed separately where possible. Use
              extended <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output to
              determine which conditions are actually pushed down. See
              <a class="xref" href="optimization.html#explain-extended" title="8.8.3 Extended EXPLAIN Output Format">Section 8.8.3, “Extended EXPLAIN Output Format”</a>.
</p></li></ul>
</div>
<p>
          Previously, condition pushdown was limited to terms referring
          to column values from the same table to which the condition
          was being pushed. Beginning with NDB 8.0.16, column values
          from tables earlier in the query plan can also be referred to
          from pushed conditions. This reduces the number of rows which
          must be handled by the SQL node during join processing.
          Filtering can be also performed in parallel in the LDM
          threads, rather than in a single <a class="link" href="programs.html#mysqld" title="4.3.1 mysqld — The MySQL Server"><span class="command"><strong>mysqld</strong></span></a>
          process. This has the potential to improve performance of
          queries by a significant margin.
        </p><p>
          Beginning with NDB 8.0.20, an outer join using a scan can be
          pushed if there are no unpushable conditions on any table used
          in the same join nest, or on any table in join nmests above it
          on which it depends. This is also true for a semijoin,
          provided the optimization strategy employed is
          <code class="literal">firstMatch</code> (see
          <a class="xref" href="optimization.html#semijoins" title="8.2.2.1 Optimizing IN and EXISTS Subquery Predicates with Semijoin Transformations">Section 8.2.2.1, “Optimizing IN and EXISTS Subquery Predicates with Semijoin
          Transformations”</a>).
        </p><p>
          Join algorithms cannot be combined with referring columns from
          previous tables in the following two situations:
</p>
<div class="orderedlist">
<ol class="orderedlist" type="1"><li class="listitem"><p>
              When any of the referred previous tables are in a join
              buffer. In this case, each row retrieved from the
              scan-filtered table is matched against every row in the
              buffer. This means that there is no single specific row
              from which column values can be fetched from when
              generating the scan filter.
            </p></li><li class="listitem"><p>
              When the column originates from a child operation in a
              pushed join. This is because rows referenced from ancestor
              operations in the join have not yet been retrieved when
              the scan filter is generated.
</p></li></ol>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="index-condition-pushdown-optimization"></a>8.2.1.6 Index Condition Pushdown Optimization</h4>

</div>

</div>

</div>
<p>
          Index Condition Pushdown (ICP) is an optimization for the case
          where MySQL retrieves rows from a table using an index.
          Without ICP, the storage engine traverses the index to locate
          rows in the base table and returns them to the MySQL server
          which evaluates the <code class="literal">WHERE</code> condition for the
          rows. With ICP enabled, and if parts of the
          <code class="literal">WHERE</code> condition can be evaluated by using
          only columns from the index, the MySQL server pushes this part
          of the <code class="literal">WHERE</code> condition down to the storage
          engine. The storage engine then evaluates the pushed index
          condition by using the index entry and only if this is
          satisfied is the row read from the table. ICP can reduce the
          number of times the storage engine must access the base table
          and the number of times the MySQL server must access the
          storage engine.
        </p><p>
          Applicability of the Index Condition Pushdown optimization is
          subject to these conditions:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              ICP is used for the
              <a class="link" href="optimization.html#jointype_range"><code class="literal">range</code></a>,
              <a class="link" href="optimization.html#jointype_ref"><code class="literal">ref</code></a>,
              <a class="link" href="optimization.html#jointype_eq_ref"><code class="literal">eq_ref</code></a>, and
              <a class="link" href="optimization.html#jointype_ref_or_null"><code class="literal">ref_or_null</code></a> access
              methods when there is a need to access full table rows.
            </p></li><li class="listitem"><p>
              ICP can be used for <a class="link" href="innodb-storage-engine.html" title="Chapter 15 The InnoDB Storage Engine"><code class="literal">InnoDB</code></a>
              and <a class="link" href="storage-engines.html#myisam-storage-engine" title="16.2 The MyISAM Storage Engine"><code class="literal">MyISAM</code></a> tables, including
              partitioned <code class="literal">InnoDB</code> and
              <code class="literal">MyISAM</code> tables.
            </p></li><li class="listitem"><p>
              For <code class="literal">InnoDB</code> tables, ICP is used only for
              secondary indexes. The goal of ICP is to reduce the number
              of full-row reads and thereby reduce I/O operations. For
              <code class="literal">InnoDB</code> clustered indexes, the complete
              record is already read into the <code class="literal">InnoDB</code>
              buffer. Using ICP in this case does not reduce I/O.
            </p></li><li class="listitem"><p>
              ICP is not supported with secondary indexes created on
              virtual generated columns. <code class="literal">InnoDB</code>
              supports secondary indexes on virtual generated columns.
            </p></li><li class="listitem"><p>
              Conditions that refer to subqueries cannot be pushed down.
            </p></li><li class="listitem"><p>
              Conditions that refer to stored functions cannot be pushed
              down. Storage engines cannot invoke stored functions.
            </p></li><li class="listitem"><p>
              Triggered conditions cannot be pushed down. (For
              information about triggered conditions, see
              <a class="xref" href="optimization.html#subquery-optimization-with-exists" title="8.2.2.3 Optimizing Subqueries with the EXISTS Strategy">Section 8.2.2.3, “Optimizing Subqueries with the EXISTS Strategy”</a>.)
</p></li></ul>
</div>
<p>
          To understand how this optimization works, first consider how
          an index scan proceeds when Index Condition Pushdown is not
          used:
</p>
<div class="orderedlist">
<ol class="orderedlist" type="1"><li class="listitem"><p>
              Get the next row, first by reading the index tuple, and
              then by using the index tuple to locate and read the full
              table row.
            </p></li><li class="listitem"><p>
              Test the part of the <code class="literal">WHERE</code> condition
              that applies to this table. Accept or reject the row based
              on the test result.
</p></li></ol>
</div>
<p>
          Using Index Condition Pushdown, the scan proceeds like this
          instead:
</p>
<div class="orderedlist">
<ol class="orderedlist" type="1"><li class="listitem"><p>
              Get the next row's index tuple (but not the full
              table row).
            </p></li><li class="listitem"><p>
              Test the part of the <code class="literal">WHERE</code> condition
              that applies to this table and can be checked using only
              index columns. If the condition is not satisfied, proceed
              to the index tuple for the next row.
            </p></li><li class="listitem"><p>
              If the condition is satisfied, use the index tuple to
              locate and read the full table row.
            </p></li><li class="listitem"><p>
              Test the remaining part of the <code class="literal">WHERE</code>
              condition that applies to this table. Accept or reject the
              row based on the test result.
</p></li></ol>
</div>
<p>
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output shows
          <code class="literal">Using index condition</code> in the
          <code class="literal">Extra</code> column when Index Condition Pushdown
          is used. It does not show <code class="literal">Using index</code>
          because that does not apply when full table rows must be read.
        </p><p>
          Suppose that a table contains information about people and
          their addresses and that the table has an index defined as
          <code class="literal">INDEX (zipcode, lastname, firstname)</code>. If we
          know a person's <code class="literal">zipcode</code> value but are
          not sure about the last name, we can search like this:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM people
  WHERE zipcode='95054'
  AND lastname LIKE '%etrunia%'
  AND address LIKE '%Main Street%';</pre><p>
          MySQL can use the index to scan through people with
          <code class="literal">zipcode='95054'</code>. The second part
          (<code class="literal">lastname LIKE '%etrunia%'</code>) cannot be used
          to limit the number of rows that must be scanned, so without
          Index Condition Pushdown, this query must retrieve full table
          rows for all people who have
          <code class="literal">zipcode='95054'</code>.
        </p><p>
          With Index Condition Pushdown, MySQL checks the
          <code class="literal">lastname LIKE '%etrunia%'</code> part before
          reading the full table row. This avoids reading full rows
          corresponding to index tuples that match the
          <code class="literal">zipcode</code> condition but not the
          <code class="literal">lastname</code> condition.
        </p><p>
          Index Condition Pushdown is enabled by default. It can be
          controlled with the
          <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
          variable by setting the
          <code class="literal">index_condition_pushdown</code> flag:
        </p><pre data-lang="sql" class="programlisting">SET optimizer_switch = 'index_condition_pushdown=off';
SET optimizer_switch = 'index_condition_pushdown=on';</pre><p>
          See <a class="xref" href="optimization.html#switchable-optimizations" title="8.9.2 Switchable Optimizations">Section 8.9.2, “Switchable Optimizations”</a>.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="nested-loop-joins"></a>8.2.1.7 Nested-Loop Join Algorithms</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444352926464"></a><a class="indexterm" name="idm46444352925408"></a><a class="indexterm" name="idm46444352923920"></a><a class="indexterm" name="idm46444352922832"></a><p>
          MySQL executes joins between tables using a nested-loop
          algorithm or variations on it.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a class="xref" href="optimization.html#nested-loop-join-algorithm" title="Nested-Loop Join Algorithm">Nested-Loop Join Algorithm</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#block-nested-loop-join-algorithm" title="Block Nested-Loop Join Algorithm">Block Nested-Loop Join Algorithm</a></p></li></ul>
</div>

<div class="simplesect">

<div class="titlepage">
<div>

<div class="simple">
<h5 class="title"><a name="nested-loop-join-algorithm"></a>Nested-Loop Join Algorithm</h5>

</div>

</div>

</div>
<p>
            A simple nested-loop join (NLJ) algorithm reads rows from
            the first table in a loop one at a time, passing each row to
            a nested loop that processes the next table in the join.
            This process is repeated as many times as there remain
            tables to be joined.
          </p><p>
            Assume that a join between three tables
            <code class="literal">t1</code>, <code class="literal">t2</code>, and
            <code class="literal">t3</code> is to be executed using the following
            join types:
          </p><pre data-lang="none" class="programlisting">Table   Join Type
t1      range
t2      ref
t3      ALL</pre><p>
            If a simple NLJ algorithm is used, the join is processed
            like this:
          </p><pre data-lang="clike" class="programlisting">for each row in t1 matching range {
  for each row in t2 matching reference key {
    for each row in t3 {
      if row satisfies join conditions, send to client
    }
  }
}</pre><p>
            Because the NLJ algorithm passes rows one at a time from
            outer loops to inner loops, it typically reads tables
            processed in the inner loops many times.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h5 class="title"><a name="block-nested-loop-join-algorithm"></a>Block Nested-Loop Join Algorithm</h5>

</div>

</div>

</div>
<p>
            A Block Nested-Loop (BNL) join algorithm uses buffering of
            rows read in outer loops to reduce the number of times that
            tables in inner loops must be read. For example, if 10 rows
            are read into a buffer and the buffer is passed to the next
            inner loop, each row read in the inner loop can be compared
            against all 10 rows in the buffer. This reduces by an order
            of magnitude the number of times the inner table must be
            read.
          </p><p>
            Prior to MySQL 8.0.18, this algorithm was applied for
            equi-joins when no indexes could be used; in MySQL 8.0.18
            and later, the hash join optimization is employed in such
            cases. Starting with MySQL 8.0.20, a hash join is also
            employed for any other join for which no indexes can be
            used, rather than the block nested-loop algorithm. See
            <a class="xref" href="optimization.html#hash-joins" title="8.2.1.4 Hash Join Optimization">Section 8.2.1.4, “Hash Join Optimization”</a>.
          </p><p>
            MySQL join buffering has these characteristics:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                Join buffering can be used when the join is of type
                <a class="link" href="optimization.html#jointype_all"><code class="literal">ALL</code></a> or
                <a class="link" href="optimization.html#jointype_index"><code class="literal">index</code></a> (in other
                words, when no possible keys can be used, and a full
                scan is done, of either the data or index rows,
                respectively), or
                <a class="link" href="optimization.html#jointype_range"><code class="literal">range</code></a>. Use of
                buffering is also applicable to outer joins, as
                described in <a class="xref" href="optimization.html#bnl-bka-optimization" title="8.2.1.12 Block Nested-Loop and Batched Key Access Joins">Section 8.2.1.12, “Block Nested-Loop and Batched Key Access Joins”</a>.
              </p></li><li class="listitem"><p>
                A join buffer is never allocated for the first
                nonconstant table, even if it would be of type
                <a class="link" href="optimization.html#jointype_all"><code class="literal">ALL</code></a> or
                <a class="link" href="optimization.html#jointype_index"><code class="literal">index</code></a>.
              </p></li><li class="listitem"><p>
                Only columns of interest to a join are stored in its
                join buffer, not whole rows.
              </p></li><li class="listitem"><p>
                The <a class="link" href="server-administration.html#sysvar_join_buffer_size"><code class="literal">join_buffer_size</code></a>
                system variable determines the size of each join buffer
                used to process a query.
              </p></li><li class="listitem"><p>
                One buffer is allocated for each join that can be
                buffered, so a given query might be processed using
                multiple join buffers.
              </p></li><li class="listitem"><p>
                A join buffer is allocated prior to executing the join
                and freed after the query is done.
</p></li></ul>
</div>
<p>
            For the example join described previously for the NLJ
            algorithm (without buffering), the join is done as follows
            using join buffering:
          </p><pre data-lang="clike" class="programlisting">for each row in t1 matching range {
  for each row in t2 matching reference key {
    store used columns from t1, t2 in join buffer
    if buffer is full {
      for each row in t3 {
        for each t1, t2 combination in join buffer {
          if row satisfies join conditions, send to client
        }
      }
      empty join buffer
    }
  }
}

if buffer is not empty {
  for each row in t3 {
    for each t1, t2 combination in join buffer {
      if row satisfies join conditions, send to client
    }
  }
}</pre><p>
            If <em class="replaceable"><code>S</code></em> is the size of each stored
            <code class="literal">t1</code>, <code class="literal">t2</code> combination in
            the join buffer and <em class="replaceable"><code>C</code></em> is the
            number of combinations in the buffer, the number of times
            table <code class="literal">t3</code> is scanned is:
          </p><pre data-lang="clike" class="programlisting">(<em class="replaceable"><code>S</code></em> * <em class="replaceable"><code>C</code></em>)/join_buffer_size + 1
</pre><p>
            The number of <code class="literal">t3</code> scans decreases as the
            value of <a class="link" href="server-administration.html#sysvar_join_buffer_size"><code class="literal">join_buffer_size</code></a>
            increases, up to the point when
            <a class="link" href="server-administration.html#sysvar_join_buffer_size"><code class="literal">join_buffer_size</code></a> is large
            enough to hold all previous row combinations. At that point,
            no speed is gained by making it larger.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="nested-join-optimization"></a>8.2.1.8 Nested Join Optimization</h4>

</div>

</div>

</div>
<p>
          The syntax for expressing joins permits nested joins. The
          following discussion refers to the join syntax described in
          <a class="xref" href="sql-statements.html#join" title="13.2.10.2 JOIN Clause">Section 13.2.10.2, “JOIN Clause”</a>.
        </p><p>
          The syntax of <em class="replaceable"><code>table_factor</code></em> is
          extended in comparison with the SQL Standard. The latter
          accepts only <em class="replaceable"><code>table_reference</code></em>, not a
          list of them inside a pair of parentheses. This is a
          conservative extension if we consider each comma in a list of
          <em class="replaceable"><code>table_reference</code></em> items as equivalent
          to an inner join. For example:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 LEFT JOIN (t2, t3, t4)
                 ON (t2.a=t1.a AND t3.b=t1.b AND t4.c=t1.c)</pre><p>
          Is equivalent to:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 LEFT JOIN (t2 CROSS JOIN t3 CROSS JOIN t4)
                 ON (t2.a=t1.a AND t3.b=t1.b AND t4.c=t1.c)</pre><p>
          In MySQL, <code class="literal">CROSS JOIN</code> is syntactically
          equivalent to <code class="literal">INNER JOIN</code>; they can replace
          each other. In standard SQL, they are not equivalent.
          <code class="literal">INNER JOIN</code> is used with an
          <code class="literal">ON</code> clause; <code class="literal">CROSS JOIN</code> is
          used otherwise.
        </p><p>
          In general, parentheses can be ignored in join expressions
          containing only inner join operations. Consider this join
          expression:
        </p><pre data-lang="sql" class="programlisting">t1 LEFT JOIN (t2 LEFT JOIN t3 ON t2.b=t3.b OR t2.b IS NULL)
   ON t1.a=t2.a</pre><p>
          After removing parentheses and grouping operations to the
          left, that join expression transforms into this expression:
        </p><pre data-lang="sql" class="programlisting">(t1 LEFT JOIN t2 ON t1.a=t2.a) LEFT JOIN t3
    ON t2.b=t3.b OR t2.b IS NULL</pre><p>
          Yet, the two expressions are not equivalent. To see this,
          suppose that the tables <code class="literal">t1</code>,
          <code class="literal">t2</code>, and <code class="literal">t3</code> have the
          following state:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Table <code class="literal">t1</code> contains rows
              <code class="literal">(1)</code>, <code class="literal">(2)</code>
            </p></li><li class="listitem"><p>
              Table <code class="literal">t2</code> contains row
              <code class="literal">(1,101)</code>
            </p></li><li class="listitem"><p>
              Table <code class="literal">t3</code> contains row
              <code class="literal">(101)</code>
</p></li></ul>
</div>
<p>
          In this case, the first expression returns a result set
          including the rows <code class="literal">(1,1,101,101)</code>,
          <code class="literal">(2,NULL,NULL,NULL)</code>, whereas the second
          expression returns the rows <code class="literal">(1,1,101,101)</code>,
          <code class="literal">(2,NULL,NULL,101)</code>:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT *</code></strong>
       <strong class="userinput"><code>FROM t1</code></strong>
            <strong class="userinput"><code>LEFT JOIN</code></strong>
            <strong class="userinput"><code>(t2 LEFT JOIN t3 ON t2.b=t3.b OR t2.b IS NULL)</code></strong>
            <strong class="userinput"><code>ON t1.a=t2.a;</code></strong>
+------+------+------+------+
| a    | a    | b    | b    |
+------+------+------+------+
|    1 |    1 |  101 |  101 |
|    2 | NULL | NULL | NULL |
+------+------+------+------+

mysql&gt; <strong class="userinput"><code>SELECT *</code></strong>
       <strong class="userinput"><code>FROM (t1 LEFT JOIN t2 ON t1.a=t2.a)</code></strong>
            <strong class="userinput"><code>LEFT JOIN t3</code></strong>
            <strong class="userinput"><code>ON t2.b=t3.b OR t2.b IS NULL;</code></strong>
+------+------+------+------+
| a    | a    | b    | b    |
+------+------+------+------+
|    1 |    1 |  101 |  101 |
|    2 | NULL | NULL |  101 |
+------+------+------+------+
</pre><p>
          In the following example, an outer join operation is used
          together with an inner join operation:
        </p><pre data-lang="sql" class="programlisting">t1 LEFT JOIN (t2, t3) ON t1.a=t2.a</pre><p>
          That expression cannot be transformed into the following
          expression:
        </p><pre data-lang="sql" class="programlisting">t1 LEFT JOIN t2 ON t1.a=t2.a, t3</pre><p>
          For the given table states, the two expressions return
          different sets of rows:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT *</code></strong>
       <strong class="userinput"><code>FROM t1 LEFT JOIN (t2, t3) ON t1.a=t2.a;</code></strong>
+------+------+------+------+
| a    | a    | b    | b    |
+------+------+------+------+
|    1 |    1 |  101 |  101 |
|    2 | NULL | NULL | NULL |
+------+------+------+------+

mysql&gt; <strong class="userinput"><code>SELECT *</code></strong>
       <strong class="userinput"><code>FROM t1 LEFT JOIN t2 ON t1.a=t2.a, t3;</code></strong>
+------+------+------+------+
| a    | a    | b    | b    |
+------+------+------+------+
|    1 |    1 |  101 |  101 |
|    2 | NULL | NULL |  101 |
+------+------+------+------+
</pre><p>
          Therefore, if we omit parentheses in a join expression with
          outer join operators, we might change the result set for the
          original expression.
        </p><p>
          More exactly, we cannot ignore parentheses in the right
          operand of the left outer join operation and in the left
          operand of a right join operation. In other words, we cannot
          ignore parentheses for the inner table expressions of outer
          join operations. Parentheses for the other operand (operand
          for the outer table) can be ignored.
        </p><p>
          The following expression:
        </p><pre data-lang="sql" class="programlisting">(t1,t2) LEFT JOIN t3 ON P(t2.b,t3.b)</pre><p>
          Is equivalent to this expression for any tables
          <code class="literal">t1,t2,t3</code> and any condition
          <code class="literal">P</code> over attributes <code class="literal">t2.b</code>
          and <code class="literal">t3.b</code>:
        </p><pre data-lang="sql" class="programlisting">t1, t2 LEFT JOIN t3 ON P(t2.b,t3.b)</pre><p>
          Whenever the order of execution of join operations in a join
          expression (<em class="replaceable"><code>joined_table</code></em>) is not
          from left to right, we talk about nested joins. Consider the
          following queries:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 LEFT JOIN (t2 LEFT JOIN t3 ON t2.b=t3.b) ON t1.a=t2.a
  WHERE t1.a &gt; 1

SELECT * FROM t1 LEFT JOIN (t2, t3) ON t1.a=t2.a
  WHERE (t2.b=t3.b OR t2.b IS NULL) AND t1.a &gt; 1</pre><p>
          Those queries are considered to contain these nested joins:
        </p><pre data-lang="sql" class="programlisting">t2 LEFT JOIN t3 ON t2.b=t3.b
t2, t3</pre><p>
          In the first query, the nested join is formed with a left join
          operation. In the second query, it is formed with an inner
          join operation.
        </p><p>
          In the first query, the parentheses can be omitted: The
          grammatical structure of the join expression will dictate the
          same order of execution for join operations. For the second
          query, the parentheses cannot be omitted, although the join
          expression here can be interpreted unambiguously without them.
          In our extended syntax, the parentheses in <code class="literal">(t2,
          t3)</code> of the second query are required, although
          theoretically the query could be parsed without them: We still
          would have unambiguous syntactical structure for the query
          because <code class="literal">LEFT JOIN</code> and <code class="literal">ON</code>
          play the role of the left and right delimiters for the
          expression <code class="literal">(t2,t3)</code>.
        </p><p>
          The preceding examples demonstrate these points:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              For join expressions involving only inner joins (and not
              outer joins), parentheses can be removed and joins
              evaluated left to right. In fact, tables can be evaluated
              in any order.
            </p></li><li class="listitem"><p>
              The same is not true, in general, for outer joins or for
              outer joins mixed with inner joins. Removal of parentheses
              may change the result.
</p></li></ul>
</div>
<a class="indexterm" name="idm46444352818848"></a><a class="indexterm" name="idm46444352817360"></a><p>
          Queries with nested outer joins are executed in the same
          pipeline manner as queries with inner joins. More exactly, a
          variation of the nested-loop join algorithm is exploited.
          Recall the algorithm by which the nested-loop join executes a
          query (see <a class="xref" href="optimization.html#nested-loop-joins" title="8.2.1.7 Nested-Loop Join Algorithms">Section 8.2.1.7, “Nested-Loop Join Algorithms”</a>). Suppose that
          a join query over 3 tables <code class="literal">T1,T2,T3</code> has
          this form:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM T1 INNER JOIN T2 ON P1(T1,T2)
                 INNER JOIN T3 ON P2(T2,T3)
  WHERE P(T1,T2,T3)</pre><p>
          Here, <code class="literal">P1(T1,T2)</code> and
          <code class="literal">P2(T3,T3)</code> are some join conditions (on
          expressions), whereas <code class="literal">P(T1,T2,T3)</code> is a
          condition over columns of tables <code class="literal">T1,T2,T3</code>.
        </p><p>
          The nested-loop join algorithm would execute this query in the
          following manner:
        </p><pre data-lang="clike" class="programlisting">FOR each row t1 in T1 {
  FOR each row t2 in T2 such that P1(t1,t2) {
    FOR each row t3 in T3 such that P2(t2,t3) {
      IF P(t1,t2,t3) {
         t:=t1||t2||t3; OUTPUT t;
      }
    }
  }
}</pre><p>
          The notation <code class="literal">t1||t2||t3</code> indicates a row
          constructed by concatenating the columns of rows
          <code class="literal">t1</code>, <code class="literal">t2</code>, and
          <code class="literal">t3</code>. In some of the following examples,
          <code class="literal">NULL</code> where a table name appears means a row
          in which <code class="literal">NULL</code> is used for each column of
          that table. For example, <code class="literal">t1||t2||NULL</code>
          indicates a row constructed by concatenating the columns of
          rows <code class="literal">t1</code> and <code class="literal">t2</code>, and
          <code class="literal">NULL</code> for each column of
          <code class="literal">t3</code>. Such a row is said to be
          <code class="literal">NULL</code>-complemented.
        </p><a class="indexterm" name="idm46444352798944"></a><p>
          Now consider a query with nested outer joins:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM T1 LEFT JOIN
              (T2 LEFT JOIN T3 ON P2(T2,T3))
              ON P1(T1,T2)
  WHERE P(T1,T2,T3)</pre><p>
          For this query, modify the nested-loop pattern to obtain:
        </p><pre data-lang="clike" class="programlisting">FOR each row t1 in T1 {
  BOOL f1:=FALSE;
  FOR each row t2 in T2 such that P1(t1,t2) {
    BOOL f2:=FALSE;
    FOR each row t3 in T3 such that P2(t2,t3) {
      IF P(t1,t2,t3) {
        t:=t1||t2||t3; OUTPUT t;
      }
      f2=TRUE;
      f1=TRUE;
    }
    IF (!f2) {
      IF P(t1,t2,NULL) {
        t:=t1||t2||NULL; OUTPUT t;
      }
      f1=TRUE;
    }
  }
  IF (!f1) {
    IF P(t1,NULL,NULL) {
      t:=t1||NULL||NULL; OUTPUT t;
    }
  }
}</pre><p>
          In general, for any nested loop for the first inner table in
          an outer join operation, a flag is introduced that is turned
          off before the loop and is checked after the loop. The flag is
          turned on when for the current row from the outer table a
          match from the table representing the inner operand is found.
          If at the end of the loop cycle the flag is still off, no
          match has been found for the current row of the outer table.
          In this case, the row is complemented by
          <code class="literal">NULL</code> values for the columns of the inner
          tables. The result row is passed to the final check for the
          output or into the next nested loop, but only if the row
          satisfies the join condition of all embedded outer joins.
        </p><p>
          In the example, the outer join table expressed by the
          following expression is embedded:
        </p><pre data-lang="sql" class="programlisting">(T2 LEFT JOIN T3 ON P2(T2,T3))</pre><p>
          For the query with inner joins, the optimizer could choose a
          different order of nested loops, such as this one:
        </p><pre data-lang="clike" class="programlisting">FOR each row t3 in T3 {
  FOR each row t2 in T2 such that P2(t2,t3) {
    FOR each row t1 in T1 such that P1(t1,t2) {
      IF P(t1,t2,t3) {
         t:=t1||t2||t3; OUTPUT t;
      }
    }
  }
}</pre><p>
          For queries with outer joins, the optimizer can choose only
          such an order where loops for outer tables precede loops for
          inner tables. Thus, for our query with outer joins, only one
          nesting order is possible. For the following query, the
          optimizer evaluates two different nestings. In both nestings,
          <code class="literal">T1</code> must be processed in the outer loop
          because it is used in an outer join. <code class="literal">T2</code> and
          <code class="literal">T3</code> are used in an inner join, so that join
          must be processed in the inner loop. However, because the join
          is an inner join, <code class="literal">T2</code> and
          <code class="literal">T3</code> can be processed in either order.
        </p><pre data-lang="sql" class="programlisting">SELECT * T1 LEFT JOIN (T2,T3) ON P1(T1,T2) AND P2(T1,T3)
  WHERE P(T1,T2,T3)</pre><p>
          One nesting evaluates <code class="literal">T2</code>, then
          <code class="literal">T3</code>:
        </p><pre data-lang="clike" class="programlisting">FOR each row t1 in T1 {
  BOOL f1:=FALSE;
  FOR each row t2 in T2 such that P1(t1,t2) {
    FOR each row t3 in T3 such that P2(t1,t3) {
      IF P(t1,t2,t3) {
        t:=t1||t2||t3; OUTPUT t;
      }
      f1:=TRUE
    }
  }
  IF (!f1) {
    IF P(t1,NULL,NULL) {
      t:=t1||NULL||NULL; OUTPUT t;
    }
  }
}</pre><p>
          The other nesting evaluates <code class="literal">T3</code>, then
          <code class="literal">T2</code>:
        </p><pre data-lang="clike" class="programlisting">FOR each row t1 in T1 {
  BOOL f1:=FALSE;
  FOR each row t3 in T3 such that P2(t1,t3) {
    FOR each row t2 in T2 such that P1(t1,t2) {
      IF P(t1,t2,t3) {
        t:=t1||t2||t3; OUTPUT t;
      }
      f1:=TRUE
    }
  }
  IF (!f1) {
    IF P(t1,NULL,NULL) {
      t:=t1||NULL||NULL; OUTPUT t;
    }
  }
}</pre><p>
          When discussing the nested-loop algorithm for inner joins, we
          omitted some details whose impact on the performance of query
          execution may be huge. We did not mention so-called
          <span class="quote">“<span class="quote">pushed-down</span>”</span> conditions. Suppose that our
          <code class="literal">WHERE</code> condition
          <code class="literal">P(T1,T2,T3)</code> can be represented by a
          conjunctive formula:
        </p><pre data-lang="clike" class="programlisting">P(T1,T2,T2) = C1(T1) AND C2(T2) AND C3(T3).</pre><p>
          In this case, MySQL actually uses the following nested-loop
          algorithm for the execution of the query with inner joins:
        </p><pre data-lang="clike" class="programlisting">FOR each row t1 in T1 such that C1(t1) {
  FOR each row t2 in T2 such that P1(t1,t2) AND C2(t2)  {
    FOR each row t3 in T3 such that P2(t2,t3) AND C3(t3) {
      IF P(t1,t2,t3) {
         t:=t1||t2||t3; OUTPUT t;
      }
    }
  }
}</pre><p>
          You see that each of the conjuncts <code class="literal">C1(T1)</code>,
          <code class="literal">C2(T2)</code>, <code class="literal">C3(T3)</code> are
          pushed out of the most inner loop to the most outer loop where
          it can be evaluated. If <code class="literal">C1(T1)</code> is a very
          restrictive condition, this condition pushdown may greatly
          reduce the number of rows from table <code class="literal">T1</code>
          passed to the inner loops. As a result, the execution time for
          the query may improve immensely.
        </p><p>
          For a query with outer joins, the <code class="literal">WHERE</code>
          condition is to be checked only after it has been found that
          the current row from the outer table has a match in the inner
          tables. Thus, the optimization of pushing conditions out of
          the inner nested loops cannot be applied directly to queries
          with outer joins. Here we must introduce conditional
          pushed-down predicates guarded by the flags that are turned on
          when a match has been encountered.
        </p><p>
          Recall this example with outer joins:
        </p><pre data-lang="clike" class="programlisting">P(T1,T2,T3)=C1(T1) AND C(T2) AND C3(T3)</pre><p>
          For that example, the nested-loop algorithm using guarded
          pushed-down conditions looks like this:
        </p><pre data-lang="clike" class="programlisting">FOR each row t1 in T1 such that C1(t1) {
  BOOL f1:=FALSE;
  FOR each row t2 in T2
      such that P1(t1,t2) AND (f1?C2(t2):TRUE) {
    BOOL f2:=FALSE;
    FOR each row t3 in T3
        such that P2(t2,t3) AND (f1&amp;&amp;f2?C3(t3):TRUE) {
      IF (f1&amp;&amp;f2?TRUE:(C2(t2) AND C3(t3))) {
        t:=t1||t2||t3; OUTPUT t;
      }
      f2=TRUE;
      f1=TRUE;
    }
    IF (!f2) {
      IF (f1?TRUE:C2(t2) &amp;&amp; P(t1,t2,NULL)) {
        t:=t1||t2||NULL; OUTPUT t;
      }
      f1=TRUE;
    }
  }
  IF (!f1 &amp;&amp; P(t1,NULL,NULL)) {
      t:=t1||NULL||NULL; OUTPUT t;
  }
}</pre><p>
          In general, pushed-down predicates can be extracted from join
          conditions such as <code class="literal">P1(T1,T2)</code> and
          <code class="literal">P(T2,T3)</code>. In this case, a pushed-down
          predicate is guarded also by a flag that prevents checking the
          predicate for the <code class="literal">NULL</code>-complemented row
          generated by the corresponding outer join operation.
        </p><p>
          Access by key from one inner table to another in the same
          nested join is prohibited if it is induced by a predicate from
          the <code class="literal">WHERE</code> condition.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="outer-join-optimization"></a>8.2.1.9 Outer Join Optimization</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444352757200"></a><a class="indexterm" name="idm46444352755712"></a><a class="indexterm" name="idm46444352754224"></a><a class="indexterm" name="idm46444352753152"></a><a class="indexterm" name="idm46444352751664"></a><a class="indexterm" name="idm46444352750592"></a><p>
          Outer joins include <code class="literal">LEFT JOIN</code> and
          <code class="literal">RIGHT JOIN</code>.
        </p><p>
          MySQL implements an <code class="literal"><em class="replaceable"><code>A</code></em> LEFT
          JOIN <em class="replaceable"><code>B</code></em>
          <em class="replaceable"><code>join_specification</code></em></code> as
          follows:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Table <em class="replaceable"><code>B</code></em> is set to depend on
              table <em class="replaceable"><code>A</code></em> and all tables on which
              <em class="replaceable"><code>A</code></em> depends.
            </p></li><li class="listitem"><p>
              Table <em class="replaceable"><code>A</code></em> is set to depend on all
              tables (except <em class="replaceable"><code>B</code></em>) that are used
              in the <code class="literal">LEFT JOIN</code> condition.
            </p></li><li class="listitem"><p>
              The <code class="literal">LEFT JOIN</code> condition is used to
              decide how to retrieve rows from table
              <em class="replaceable"><code>B</code></em>. (In other words, any
              condition in the <code class="literal">WHERE</code> clause is not
              used.)
            </p></li><li class="listitem"><p>
              All standard join optimizations are performed, with the
              exception that a table is always read after all tables on
              which it depends. If there is a circular dependency, an
              error occurs.
            </p></li><li class="listitem"><p>
              All standard <code class="literal">WHERE</code> optimizations are
              performed.
            </p></li><li class="listitem"><p>
              If there is a row in <em class="replaceable"><code>A</code></em> that
              matches the <code class="literal">WHERE</code> clause, but there is
              no row in <em class="replaceable"><code>B</code></em> that matches the
              <code class="literal">ON</code> condition, an extra
              <em class="replaceable"><code>B</code></em> row is generated with all
              columns set to <code class="literal">NULL</code>.
            </p></li><li class="listitem"><p>
              If you use <code class="literal">LEFT JOIN</code> to find rows that
              do not exist in some table and you have the following
              test: <code class="literal"><em class="replaceable"><code>col_name</code></em> IS
              NULL</code> in the <code class="literal">WHERE</code> part, where
              <em class="replaceable"><code>col_name</code></em> is a column that is
              declared as <code class="literal">NOT NULL</code>, MySQL stops
              searching for more rows (for a particular key combination)
              after it has found one row that matches the <code class="literal">LEFT
              JOIN</code> condition.
</p></li></ul>
</div>
<p>
          The <code class="literal">RIGHT JOIN</code> implementation is analogous
          to that of <code class="literal">LEFT JOIN</code> with the table roles
          reversed. Right joins are converted to equivalent left joins,
          as described in <a class="xref" href="optimization.html#outer-join-simplification" title="8.2.1.10 Outer Join Simplification">Section 8.2.1.10, “Outer Join Simplification”</a>.
        </p><p>
          For a <code class="literal">LEFT JOIN</code>, if the
          <code class="literal">WHERE</code> condition is always false for the
          generated <code class="literal">NULL</code> row, the <code class="literal">LEFT
          JOIN</code> is changed to an inner join. For example, the
          <code class="literal">WHERE</code> clause would be false in the
          following query if <code class="literal">t2.column1</code> were
          <code class="literal">NULL</code>:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 LEFT JOIN t2 ON (column1) WHERE t2.column2=5;</pre><p>
          Therefore, it is safe to convert the query to an inner join:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1, t2 WHERE t2.column2=5 AND t1.column1=t2.column1;</pre><p>
          In MySQL 8.0.14 and later, trivial <code class="literal">WHERE</code>
          conditions arising from constant literal expressions are
          removed during preparation, rather than at a later stage in
          optimization, by which time joins have already been
          simplified. Earlier removal of trivial conditions allows the
          optimizer to convert outer joins to inner joins; this can
          result in improved plans for queries with outer joins
          containing trivial conditions in the <code class="literal">WHERE</code>
          clause, such as this one:
        </p><pre class="programlisting">SELECT * FROM t1 LEFT JOIN t2 ON <em class="replaceable"><code>condition_1</code></em> WHERE <em class="replaceable"><code>condition_2</code></em> OR 0 = 1
</pre><p>
          The optimizer now sees during preparation that 0 = 1 is always
          false, making <code class="literal">OR 0 = 1</code> redundant, and
          removes it, leaving this:
        </p><pre class="programlisting">SELECT * FROM t1 LEFT JOIN t2 ON <em class="replaceable"><code>condition_1</code></em> where <em class="replaceable"><code>condition_2</code></em>
</pre><p>
          Now the optimizer can rewrite the query as an inner join, like
          this:
        </p><pre class="programlisting">SELECT * FROM t1 JOIN t2 WHERE <em class="replaceable"><code>condition_1</code></em> AND <em class="replaceable"><code>condition_2</code></em>
</pre><a class="indexterm" name="idm46444352705216"></a><p>
          Now the optimizer can use table <code class="literal">t2</code> before
          table <code class="literal">t1</code> if doing so would result in a
          better query plan. To provide a hint about the table join
          order, use optimizer hints; see
          <a class="xref" href="optimization.html#optimizer-hints" title="8.9.3 Optimizer Hints">Section 8.9.3, “Optimizer Hints”</a>. Alternatively, use
          <code class="literal">STRAIGHT_JOIN</code>; see
          <a class="xref" href="sql-statements.html#select" title="13.2.10 SELECT Statement">Section 13.2.10, “SELECT Statement”</a>. However,
          <code class="literal">STRAIGHT_JOIN</code> may prevent indexes from
          being used because it disables semijoin transformations; see
          <a class="xref" href="optimization.html#semijoins" title="8.2.2.1 Optimizing IN and EXISTS Subquery Predicates with Semijoin Transformations">Section 8.2.2.1, “Optimizing IN and EXISTS Subquery Predicates with Semijoin
          Transformations”</a>.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="outer-join-simplification"></a>8.2.1.10 Outer Join Simplification</h4>

</div>

</div>

</div>
<p>
          Table expressions in the <code class="literal">FROM</code> clause of a
          query are simplified in many cases.
        </p><p>
          At the parser stage, queries with right outer join operations
          are converted to equivalent queries containing only left join
          operations. In the general case, the conversion is performed
          such that this right join:
        </p><pre data-lang="sql" class="programlisting">(T1, ...) RIGHT JOIN (T2, ...) ON P(T1, ..., T2, ...)</pre><p>
          Becomes this equivalent left join:
        </p><pre data-lang="sql" class="programlisting">(T2, ...) LEFT JOIN (T1, ...) ON P(T1, ..., T2, ...)</pre><p>
          All inner join expressions of the form <code class="literal">T1 INNER JOIN
          T2 ON P(T1,T2)</code> are replaced by the list
          <code class="literal">T1,T2</code>, <code class="literal">P(T1,T2)</code> being
          joined as a conjunct to the <code class="literal">WHERE</code> condition
          (or to the join condition of the embedding join, if there is
          any).
        </p><p>
          When the optimizer evaluates plans for outer join operations,
          it takes into consideration only plans where, for each such
          operation, the outer tables are accessed before the inner
          tables. The optimizer choices are limited because only such
          plans enable outer joins to be executed using the nested-loop
          algorithm.
        </p><p>
          Consider a query of this form, where <code class="literal">R(T2)</code>
          greatly narrows the number of matching rows from table
          <code class="literal">T2</code>:
        </p><pre data-lang="sql" class="programlisting">SELECT * T1 LEFT JOIN T2 ON P1(T1,T2)
  WHERE P(T1,T2) AND R(T2)</pre><p>
          If the query is executed as written, the optimizer has no
          choice but to access the less-restricted table
          <code class="literal">T1</code> before the more-restricted table
          <code class="literal">T2</code>, which may produce a very inefficient
          execution plan.
        </p><p>
          Instead, MySQL converts the query to a query with no outer
          join operation if the <code class="literal">WHERE</code> condition is
          null-rejected. (That is, it converts the outer join to an
          inner join.) A condition is said to be null-rejected for an
          outer join operation if it evaluates to
          <code class="literal">FALSE</code> or <code class="literal">UNKNOWN</code> for any
          <code class="literal">NULL</code>-complemented row generated for the
          operation.
        </p><a class="indexterm" name="idm46444352680400"></a><a class="indexterm" name="idm46444352679328"></a><p>
          Thus, for this outer join:
        </p><pre data-lang="sql" class="programlisting">T1 LEFT JOIN T2 ON T1.A=T2.A</pre><p>
          Conditions such as these are null-rejected because they cannot
          be true for any <code class="literal">NULL</code>-complemented row (with
          <code class="literal">T2</code> columns set to <code class="literal">NULL</code>):
        </p><pre data-lang="sql" class="programlisting">T2.B IS NOT NULL
T2.B &gt; 3
T2.C &lt;= T1.C
T2.B &lt; 2 OR T2.C &gt; 1</pre><p>
          Conditions such as these are not null-rejected because they
          might be true for a <code class="literal">NULL</code>-complemented row:
        </p><pre data-lang="sql" class="programlisting">T2.B IS NULL
T1.B &lt; 3 OR T2.B IS NOT NULL
T1.B &lt; 3 OR T2.B &gt; 3</pre><p>
          The general rules for checking whether a condition is
          null-rejected for an outer join operation are simple:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              It is of the form <code class="literal">A IS NOT NULL</code>, where
              <code class="literal">A</code> is an attribute of any of the inner
              tables
            </p></li><li class="listitem"><p>
              It is a predicate containing a reference to an inner table
              that evaluates to <code class="literal">UNKNOWN</code> when one of
              its arguments is <code class="literal">NULL</code>
            </p></li><li class="listitem"><p>
              It is a conjunction containing a null-rejected condition
              as a conjunct
            </p></li><li class="listitem"><p>
              It is a disjunction of null-rejected conditions
</p></li></ul>
</div>
<p>
          A condition can be null-rejected for one outer join operation
          in a query and not null-rejected for another. In this query,
          the <code class="literal">WHERE</code> condition is null-rejected for
          the second outer join operation but is not null-rejected for
          the first one:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM T1 LEFT JOIN T2 ON T2.A=T1.A
                 LEFT JOIN T3 ON T3.B=T1.B
  WHERE T3.C &gt; 0</pre><p>
          If the <code class="literal">WHERE</code> condition is null-rejected for
          an outer join operation in a query, the outer join operation
          is replaced by an inner join operation.
        </p><p>
          For example, in the preceding query, the second outer join is
          null-rejected and can be replaced by an inner join:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM T1 LEFT JOIN T2 ON T2.A=T1.A
                 INNER JOIN T3 ON T3.B=T1.B
  WHERE T3.C &gt; 0</pre><p>
          For the original query, the optimizer evaluates only plans
          compatible with the single table-access order
          <code class="literal">T1,T2,T3</code>. For the rewritten query, it
          additionally considers the access order
          <code class="literal">T3,T1,T2</code>.
        </p><p>
          A conversion of one outer join operation may trigger a
          conversion of another. Thus, the query:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM T1 LEFT JOIN T2 ON T2.A=T1.A
                 LEFT JOIN T3 ON T3.B=T2.B
  WHERE T3.C &gt; 0</pre><p>
          Is first converted to the query:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM T1 LEFT JOIN T2 ON T2.A=T1.A
                 INNER JOIN T3 ON T3.B=T2.B
  WHERE T3.C &gt; 0</pre><p>
          Which is equivalent to the query:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM (T1 LEFT JOIN T2 ON T2.A=T1.A), T3
  WHERE T3.C &gt; 0 AND T3.B=T2.B</pre><p>
          The remaining outer join operation can also be replaced by an
          inner join because the condition <code class="literal">T3.B=T2.B</code>
          is null-rejected. This results in a query with no outer joins
          at all:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM (T1 INNER JOIN T2 ON T2.A=T1.A), T3
  WHERE T3.C &gt; 0 AND T3.B=T2.B</pre><p>
          Sometimes the optimizer succeeds in replacing an embedded
          outer join operation, but cannot convert the embedding outer
          join. The following query:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM T1 LEFT JOIN
              (T2 LEFT JOIN T3 ON T3.B=T2.B)
              ON T2.A=T1.A
  WHERE T3.C &gt; 0</pre><p>
          Is converted to:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM T1 LEFT JOIN
              (T2 INNER JOIN T3 ON T3.B=T2.B)
              ON T2.A=T1.A
  WHERE T3.C &gt; 0</pre><p>
          That can be rewritten only to the form still containing the
          embedding outer join operation:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM T1 LEFT JOIN
              (T2,T3)
              ON (T2.A=T1.A AND T3.B=T2.B)
  WHERE T3.C &gt; 0</pre><p>
          Any attempt to convert an embedded outer join operation in a
          query must take into account the join condition for the
          embedding outer join together with the
          <code class="literal">WHERE</code> condition. In this query, the
          <code class="literal">WHERE</code> condition is not null-rejected for
          the embedded outer join, but the join condition of the
          embedding outer join <code class="literal">T2.A=T1.A AND
          T3.C=T1.C</code> is null-rejected:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM T1 LEFT JOIN
              (T2 LEFT JOIN T3 ON T3.B=T2.B)
              ON T2.A=T1.A AND T3.C=T1.C
  WHERE T3.D &gt; 0 OR T1.D &gt; 0</pre><p>
          Consequently, the query can be converted to:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM T1 LEFT JOIN
              (T2, T3)
              ON T2.A=T1.A AND T3.C=T1.C AND T3.B=T2.B
WHERE T3.D &gt; 0 OR T1.D &gt; 0</pre>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="mrr-optimization"></a>8.2.1.11 Multi-Range Read Optimization</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444352637104"></a><a class="indexterm" name="idm46444352635648"></a><p>
          Reading rows using a range scan on a secondary index can
          result in many random disk accesses to the base table when the
          table is large and not stored in the storage engine's
          cache. With the Disk-Sweep Multi-Range Read (MRR)
          optimization, MySQL tries to reduce the number of random disk
          access for range scans by first scanning the index only and
          collecting the keys for the relevant rows. Then the keys are
          sorted and finally the rows are retrieved from the base table
          using the order of the primary key. The motivation for
          Disk-sweep MRR is to reduce the number of random disk accesses
          and instead achieve a more sequential scan of the base table
          data.
        </p><p>
          The Multi-Range Read optimization provides these benefits:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              MRR enables data rows to be accessed sequentially rather
              than in random order, based on index tuples. The server
              obtains a set of index tuples that satisfy the query
              conditions, sorts them according to data row ID order, and
              uses the sorted tuples to retrieve data rows in order.
              This makes data access more efficient and less expensive.
            </p></li><li class="listitem"><p>
              MRR enables batch processing of requests for key access
              for operations that require access to data rows through
              index tuples, such as range index scans and equi-joins
              that use an index for the join attribute. MRR iterates
              over a sequence of index ranges to obtain qualifying index
              tuples. As these results accumulate, they are used to
              access the corresponding data rows. It is not necessary to
              acquire all index tuples before starting to read data
              rows.
</p></li></ul>
</div>
<p>
          The MRR optimization is not supported with secondary indexes
          created on virtual generated columns.
          <code class="literal">InnoDB</code> supports secondary indexes on
          virtual generated columns.
        </p><p>
          The following scenarios illustrate when MRR optimization can
          be advantageous:
        </p><p>
          Scenario A: MRR can be used for <code class="literal">InnoDB</code> and
          <code class="literal">MyISAM</code> tables for index range scans and
          equi-join operations.
</p>
<div class="orderedlist">
<ol class="orderedlist" type="1"><li class="listitem"><p>
              A portion of the index tuples are accumulated in a buffer.
            </p></li><li class="listitem"><p>
              The tuples in the buffer are sorted by their data row ID.
            </p></li><li class="listitem"><p>
              Data rows are accessed according to the sorted index tuple
              sequence.
</p></li></ol>
</div>
<p>
          Scenario B: MRR can be used for
          <a class="link" href="mysql-cluster.html" title="Chapter 22 MySQL NDB Cluster 8.0"><code class="literal">NDB</code></a> tables for multiple-range
          index scans or when performing an equi-join by an attribute.
</p>
<div class="orderedlist">
<ol class="orderedlist" type="1"><li class="listitem"><p>
              A portion of ranges, possibly single-key ranges, is
              accumulated in a buffer on the central node where the
              query is submitted.
            </p></li><li class="listitem"><p>
              The ranges are sent to the execution nodes that access
              data rows.
            </p></li><li class="listitem"><p>
              The accessed rows are packed into packages and sent back
              to the central node.
            </p></li><li class="listitem"><p>
              The received packages with data rows are placed in a
              buffer.
            </p></li><li class="listitem"><p>
              Data rows are read from the buffer.
</p></li></ol>
</div>
<p>
          When MRR is used, the <code class="literal">Extra</code> column in
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output shows
          <code class="literal">Using MRR</code>.
        </p><p>
          <code class="literal">InnoDB</code> and <code class="literal">MyISAM</code> do not
          use MRR if full table rows need not be accessed to produce the
          query result. This is the case if results can be produced
          entirely on the basis on information in the index tuples
          (through a <a class="link" href="glossary.html#glos_covering_index" title="covering index">covering
          index</a>); MRR provides no benefit.
        </p><p>
          Two <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
          variable flags provide an interface to the use of MRR
          optimization. The <code class="literal">mrr</code> flag controls whether
          MRR is enabled. If <code class="literal">mrr</code> is enabled
          (<code class="literal">on</code>), the <code class="literal">mrr_cost_based</code>
          flag controls whether the optimizer attempts to make a
          cost-based choice between using and not using MRR
          (<code class="literal">on</code>) or uses MRR whenever possible
          (<code class="literal">off</code>). By default, <code class="literal">mrr</code>
          is <code class="literal">on</code> and <code class="literal">mrr_cost_based</code>
          is <code class="literal">on</code>. See
          <a class="xref" href="optimization.html#switchable-optimizations" title="8.9.2 Switchable Optimizations">Section 8.9.2, “Switchable Optimizations”</a>.
        </p><p>
          For MRR, a storage engine uses the value of the
          <a class="link" href="server-administration.html#sysvar_read_rnd_buffer_size"><code class="literal">read_rnd_buffer_size</code></a> system
          variable as a guideline for how much memory it can allocate
          for its buffer. The engine uses up to
          <a class="link" href="server-administration.html#sysvar_read_rnd_buffer_size"><code class="literal">read_rnd_buffer_size</code></a> bytes
          and determines the number of ranges to process in a single
          pass.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="bnl-bka-optimization"></a>8.2.1.12 Block Nested-Loop and Batched Key Access Joins</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444352594672"></a><a class="indexterm" name="idm46444352593216"></a><a class="indexterm" name="idm46444352591728"></a><a class="indexterm" name="idm46444352590240"></a><p>
          In MySQL, a Batched Key Access (BKA) Join algorithm is
          available that uses both index access to the joined table and
          a join buffer. The BKA algorithm supports inner join, outer
          join, and semijoin operations, including nested outer joins.
          Benefits of BKA include improved join performance due to more
          efficient table scanning. Also, the Block Nested-Loop (BNL)
          Join algorithm previously used only for inner joins is
          extended and can be employed for outer join and semijoin
          operations, including nested outer joins.
        </p><p>
          The following sections discuss the join buffer management that
          underlies the extension of the original BNL algorithm, the
          extended BNL algorithm, and the BKA algorithm. For information
          about semijoin strategies, see <a class="xref" href="optimization.html#semijoins" title="8.2.2.1 Optimizing IN and EXISTS Subquery Predicates with Semijoin Transformations">Section 8.2.2.1, “Optimizing IN and EXISTS Subquery Predicates with Semijoin
          Transformations”</a>
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a class="xref" href="optimization.html#join-buffer-management" title="Join Buffer Management for Block Nested-Loop and Batched Key Access Algorithms">Join Buffer Management for Block Nested-Loop and Batched Key Access
Algorithms</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#bnl-optimization" title="Block Nested-Loop Algorithm for Outer Joins and Semijoins">Block Nested-Loop Algorithm for Outer Joins and Semijoins</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#bka-optimization" title="Batched Key Access Joins">Batched Key Access Joins</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#bnl-bka-optimizer-hints" title="Optimizer Hints for Block Nested-Loop and Batched Key Access Algorithms">Optimizer Hints for Block Nested-Loop and Batched Key Access Algorithms</a></p></li></ul>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h5 class="title"><a name="join-buffer-management"></a>Join Buffer Management for Block Nested-Loop and Batched Key Access
Algorithms</h5>
</div>
</div>
</div>
<p>
            MySQL can employ join buffers to execute not only inner
            joins without index access to the inner table, but also
            outer joins and semijoins that appear after subquery
            flattening. Moreover, a join buffer can be effectively used
            when there is an index access to the inner table.
          </p><p>
            The join buffer management code slightly more efficiently
            utilizes join buffer space when storing the values of the
            interesting row columns: No additional bytes are allocated
            in buffers for a row column if its value is
            <code class="literal">NULL</code>, and the minimum number of bytes is
            allocated for any value of the
            <a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">VARCHAR</code></a> type.
          </p><p>
            The code supports two types of buffers, regular and
            incremental. Suppose that join buffer <code class="literal">B1</code>
            is employed to join tables <code class="literal">t1</code> and
            <code class="literal">t2</code> and the result of this operation is
            joined with table <code class="literal">t3</code> using join buffer
            <code class="literal">B2</code>:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                A regular join buffer contains columns from each join
                operand. If <code class="literal">B2</code> is a regular join
                buffer, each row <em class="replaceable"><code>r</code></em> put into
                <code class="literal">B2</code> is composed of the columns of a
                row <em class="replaceable"><code>r1</code></em> from
                <code class="literal">B1</code> and the interesting columns of a
                matching row <em class="replaceable"><code>r2</code></em> from table
                <code class="literal">t3</code>.
              </p></li><li class="listitem"><p>
                An incremental join buffer contains only columns from
                rows of the table produced by the second join operand.
                That is, it is incremental to a row from the first
                operand buffer. If <code class="literal">B2</code> is an
                incremental join buffer, it contains the interesting
                columns of the row <em class="replaceable"><code>r2</code></em>
                together with a link to the row
                <em class="replaceable"><code>r1</code></em> from
                <code class="literal">B1</code>.
</p></li></ul>
</div>
<p>
            Incremental join buffers are always incremental relative to
            a join buffer from an earlier join operation, so the buffer
            from the first join operation is always a regular buffer. In
            the example just given, the buffer <code class="literal">B1</code>
            used to join tables <code class="literal">t1</code> and
            <code class="literal">t2</code> must be a regular buffer.
          </p><p>
            Each row of the incremental buffer used for a join operation
            contains only the interesting columns of a row from the
            table to be joined. These columns are augmented with a
            reference to the interesting columns of the matched row from
            the table produced by the first join operand. Several rows
            in the incremental buffer can refer to the same row
            <em class="replaceable"><code>r</code></em> whose columns are stored in the
            previous join buffers insofar as all these rows match row
            <em class="replaceable"><code>r</code></em>.
          </p><p>
            Incremental buffers enable less frequent copying of columns
            from buffers used for previous join operations. This
            provides a savings in buffer space because in the general
            case a row produced by the first join operand can be matched
            by several rows produced by the second join operand. It is
            unnecessary to make several copies of a row from the first
            operand. Incremental buffers also provide a savings in
            processing time due to the reduction in copying time.
          </p><p>
            The <code class="literal">block_nested_loop</code> and
            <code class="literal">batched_key_access</code> flags of the
            <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
            variable control how the optimizer uses the Block
            Nested-Loop and Batched Key Access join algorithms. By
            default, <code class="literal">block_nested_loop</code> is
            <code class="literal">on</code> and
            <code class="literal">batched_key_access</code> is
            <code class="literal">off</code>. See
            <a class="xref" href="optimization.html#switchable-optimizations" title="8.9.2 Switchable Optimizations">Section 8.9.2, “Switchable Optimizations”</a>. Optimizer hints
            may also be applied; see
            <a class="xref" href="optimization.html#bnl-bka-optimizer-hints" title="Optimizer Hints for Block Nested-Loop and Batched Key Access Algorithms">Optimizer Hints for Block Nested-Loop and Batched Key Access Algorithms</a>.
          </p><p>
            For information about semijoin strategies, see
            <a class="xref" href="optimization.html#semijoins" title="8.2.2.1 Optimizing IN and EXISTS Subquery Predicates with Semijoin Transformations">Section 8.2.2.1, “Optimizing IN and EXISTS Subquery Predicates with Semijoin
          Transformations”</a>
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h5 class="title"><a name="bnl-optimization"></a>Block Nested-Loop Algorithm for Outer Joins and Semijoins</h5>

</div>

</div>

</div>
<a class="indexterm" name="idm46444352548688"></a><a class="indexterm" name="idm46444352547232"></a><p>
            The original implementation of the MySQL BNL algorithm is
            extended to support outer join and semijoin operations.
          </p><p>
            When these operations are executed with a join buffer, each
            row put into the buffer is supplied with a match flag.
          </p><p>
            If an outer join operation is executed using a join buffer,
            each row of the table produced by the second operand is
            checked for a match against each row in the join buffer.
            When a match is found, a new extended row is formed (the
            original row plus columns from the second operand) and sent
            for further extensions by the remaining join operations. In
            addition, the match flag of the matched row in the buffer is
            enabled. After all rows of the table to be joined have been
            examined, the join buffer is scanned. Each row from the
            buffer that does not have its match flag enabled is extended
            by <code class="literal">NULL</code> complements
            (<code class="literal">NULL</code> values for each column in the
            second operand) and sent for further extensions by the
            remaining join operations.
          </p><p>
            The <code class="literal">block_nested_loop</code> flag of the
            <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
            variable controls how the optimizer uses the Block
            Nested-Loop algorithm. By default,
            <code class="literal">block_nested_loop</code> is
            <code class="literal">on</code>. See
            <a class="xref" href="optimization.html#switchable-optimizations" title="8.9.2 Switchable Optimizations">Section 8.9.2, “Switchable Optimizations”</a>. Optimizer hints
            may also be applied; see
            <a class="xref" href="optimization.html#bnl-bka-optimizer-hints" title="Optimizer Hints for Block Nested-Loop and Batched Key Access Algorithms">Optimizer Hints for Block Nested-Loop and Batched Key Access Algorithms</a>.
          </p><p>
            In <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output, use of BNL
            for a table is signified when the <code class="literal">Extra</code>
            value contains <code class="literal">Using join buffer (Block Nested
            Loop)</code> and the <code class="literal">type</code> value is
            <a class="link" href="optimization.html#jointype_all"><code class="literal">ALL</code></a>,
            <a class="link" href="optimization.html#jointype_index"><code class="literal">index</code></a>, or
            <a class="link" href="optimization.html#jointype_range"><code class="literal">range</code></a>.
          </p><p>
            For information about semijoin strategies, see
            <a class="xref" href="optimization.html#semijoins" title="8.2.2.1 Optimizing IN and EXISTS Subquery Predicates with Semijoin Transformations">Section 8.2.2.1, “Optimizing IN and EXISTS Subquery Predicates with Semijoin
          Transformations”</a>
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h5 class="title"><a name="bka-optimization"></a>Batched Key Access Joins</h5>

</div>

</div>

</div>
<a class="indexterm" name="idm46444352527392"></a><a class="indexterm" name="idm46444352525936"></a><p>
            MySQL implements a method of joining tables called the
            Batched Key Access (BKA) join algorithm. BKA can be applied
            when there is an index access to the table produced by the
            second join operand. Like the BNL join algorithm, the BKA
            join algorithm employs a join buffer to accumulate the
            interesting columns of the rows produced by the first
            operand of the join operation. Then the BKA algorithm builds
            keys to access the table to be joined for all rows in the
            buffer and submits these keys in a batch to the database
            engine for index lookups. The keys are submitted to the
            engine through the Multi-Range Read (MRR) interface (see
            <a class="xref" href="optimization.html#mrr-optimization" title="8.2.1.11 Multi-Range Read Optimization">Section 8.2.1.11, “Multi-Range Read Optimization”</a>). After submission of the
            keys, the MRR engine functions perform lookups in the index
            in an optimal way, fetching the rows of the joined table
            found by these keys, and starts feeding the BKA join
            algorithm with matching rows. Each matching row is coupled
            with a reference to a row in the join buffer.
          </p><p>
            When BKA is used, the value of
            <a class="link" href="server-administration.html#sysvar_join_buffer_size"><code class="literal">join_buffer_size</code></a> defines
            how large the batch of keys is in each request to the
            storage engine. The larger the buffer, the more sequential
            access will be to the right hand table of a join operation,
            which can significantly improve performance.
          </p><p>
            For BKA to be used, the
            <code class="literal">batched_key_access</code> flag of the
            <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
            variable must be set to <code class="literal">on</code>. BKA uses MRR,
            so the <code class="literal">mrr</code> flag must also be
            <code class="literal">on</code>. Currently, the cost estimation for
            MRR is too pessimistic. Hence, it is also necessary for
            <code class="literal">mrr_cost_based</code> to be
            <code class="literal">off</code> for BKA to be used. The following
            setting enables BKA:
          </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SET optimizer_switch='mrr=on,mrr_cost_based=off,batched_key_access=on';</code></strong>
</pre><p>
            There are two scenarios by which MRR functions execute:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                The first scenario is used for conventional disk-based
                storage engines such as
                <a class="link" href="innodb-storage-engine.html" title="Chapter 15 The InnoDB Storage Engine"><code class="literal">InnoDB</code></a> and
                <a class="link" href="storage-engines.html#myisam-storage-engine" title="16.2 The MyISAM Storage Engine"><code class="literal">MyISAM</code></a>. For these engines,
                usually the keys for all rows from the join buffer are
                submitted to the MRR interface at once. Engine-specific
                MRR functions perform index lookups for the submitted
                keys, get row IDs (or primary keys) from them, and then
                fetch rows for all these selected row IDs one by one by
                request from BKA algorithm. Every row is returned with
                an association reference that enables access to the
                matched row in the join buffer. The rows are fetched by
                the MRR functions in an optimal way: They are fetched in
                the row ID (primary key) order. This improves
                performance because reads are in disk order rather than
                random order.
              </p></li><li class="listitem"><p>
                The second scenario is used for remote storage engines
                such as <a class="link" href="mysql-cluster.html" title="Chapter 22 MySQL NDB Cluster 8.0"><code class="literal">NDB</code></a>. A package of
                keys for a portion of rows from the join buffer,
                together with their associations, is sent by a MySQL
                Server (SQL node) to MySQL Cluster data nodes. In
                return, the SQL node receives a package (or several
                packages) of matching rows coupled with corresponding
                associations. The BKA join algorithm takes these rows
                and builds new joined rows. Then a new set of keys is
                sent to the data nodes and the rows from the returned
                packages are used to build new joined rows. The process
                continues until the last keys from the join buffer are
                sent to the data nodes, and the SQL node has received
                and joined all rows matching these keys. This improves
                performance because fewer key-bearing packages sent by
                the SQL node to the data nodes means fewer round trips
                between it and the data nodes to perform the join
                operation.
</p></li></ul>
</div>
<p>
            With the first scenario, a portion of the join buffer is
            reserved to store row IDs (primary keys) selected by index
            lookups and passed as a parameter to the MRR functions.
          </p><p>
            There is no special buffer to store keys built for rows from
            the join buffer. Instead, a function that builds the key for
            the next row in the buffer is passed as a parameter to the
            MRR functions.
          </p><p>
            In <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output, use of BKA
            for a table is signified when the <code class="literal">Extra</code>
            value contains <code class="literal">Using join buffer (Batched Key
            Access)</code> and the <code class="literal">type</code> value is
            <a class="link" href="optimization.html#jointype_ref"><code class="literal">ref</code></a> or
            <a class="link" href="optimization.html#jointype_eq_ref"><code class="literal">eq_ref</code></a>.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h5 class="title"><a name="bnl-bka-optimizer-hints"></a>Optimizer Hints for Block Nested-Loop and Batched Key Access Algorithms</h5>

</div>

</div>

</div>
<p>
            In addition to using the
            <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
            variable to control optimizer use of the BNL and BKA
            algorithms session-wide, MySQL supports optimizer hints to
            influence the optimizer on a per-statement basis. See
            <a class="xref" href="optimization.html#optimizer-hints" title="8.9.3 Optimizer Hints">Section 8.9.3, “Optimizer Hints”</a>.
          </p><p>
            To use a BNL or BKA hint to enable join buffering for any
            inner table of an outer join, join buffering must be enabled
            for all inner tables of the outer join.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="condition-filtering"></a>8.2.1.13 Condition Filtering</h4>

</div>

</div>

</div>
<p>
          In join processing, prefix rows are those rows passed from one
          table in a join to the next. In general, the optimizer
          attempts to put tables with low prefix counts early in the
          join order to keep the number of row combinations from
          increasing rapidly. To the extent that the optimizer can use
          information about conditions on rows selected from one table
          and passed to the next, the more accurately it can compute row
          estimates and choose the best execution plan.
        </p><p>
          Without condition filtering, the prefix row count for a table
          is based on the estimated number of rows selected by the
          <code class="literal">WHERE</code> clause according to whichever access
          method the optimizer chooses. Condition filtering enables the
          optimizer to use other relevant conditions in the
          <code class="literal">WHERE</code> clause not taken into account by the
          access method, and thus improve its prefix row count
          estimates. For example, even though there might be an
          index-based access method that can be used to select rows from
          the current table in a join, there might also be additional
          conditions for the table in the <code class="literal">WHERE</code>
          clause that can filter (further restrict) the estimate for
          qualifying rows passed to the next table.
        </p><p>
          A condition contributes to the filtering estimate only if:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              It refers to the current table.
            </p></li><li class="listitem"><p>
              It depends on a constant value or values from earlier
              tables in the join sequence.
            </p></li><li class="listitem"><p>
              It was not already taken into account by the access
              method.
</p></li></ul>
</div>
<p>
          In <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output, the
          <code class="literal">rows</code> column indicates the row estimate for
          the chosen access method, and the <code class="literal">filtered</code>
          column reflects the effect of condition filtering.
          <code class="literal">filtered</code> values are expressed as
          percentages. The maximum value is 100, which means no
          filtering of rows occurred. Values decreasing from 100
          indicate increasing amounts of filtering.
        </p><p>
          The prefix row count (the number of rows estimated to be
          passed from the current table in a join to the next) is the
          product of the <code class="literal">rows</code> and
          <code class="literal">filtered</code> values. That is, the prefix row
          count is the estimated row count, reduced by the estimated
          filtering effect. For example, if <code class="literal">rows</code> is
          1000 and <code class="literal">filtered</code> is 20%, condition
          filtering reduces the estimated row count of 1000 to a prefix
          row count of 1000 × 20% = 1000 × .2 = 200.
        </p><p>
          Consider the following query:
        </p><pre data-lang="sql" class="programlisting">SELECT *
  FROM employee JOIN department ON employee.dept_no = department.dept_no
  WHERE employee.first_name = 'John'
  AND employee.hire_date BETWEEN '2018-01-01' AND '2018-06-01';</pre><p>
          Suppose that the data set has these characteristics:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              The <code class="literal">employee</code> table has 1024 rows.
            </p></li><li class="listitem"><p>
              The <code class="literal">department</code> table has 12 rows.
            </p></li><li class="listitem"><p>
              Both tables have an index on <code class="literal">dept_no</code>.
            </p></li><li class="listitem"><p>
              The <code class="literal">employee</code> table has an index on
              <code class="literal">first_name</code>.
            </p></li><li class="listitem"><p>
              8 rows satisfy this condition on
              <code class="literal">employee.first_name</code>:
            </p><pre data-lang="sql" class="programlisting">employee.first_name = 'John'</pre></li><li class="listitem"><p>
              150 rows satisfy this condition on
              <code class="literal">employee.hire_date</code>:
            </p><pre data-lang="sql" class="programlisting">employee.hire_date BETWEEN '2018-01-01' AND '2018-06-01'</pre></li><li class="listitem"><p>
              1 row satisfies both conditions:
            </p><pre data-lang="sql" class="programlisting">employee.first_name = 'John'
AND employee.hire_date BETWEEN '2018-01-01' AND '2018-06-01'</pre></li></ul>
</div>
<p>
          Without condition filtering,
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> produces output like
          this:
        </p><pre data-lang="none" class="programlisting">+----+------------+--------+------------------+---------+---------+------+----------+
| id | table      | type   | possible_keys    | key     | ref     | rows | filtered |
+----+------------+--------+------------------+---------+---------+------+----------+
| 1  | employee   | ref    | name,h_date,dept | name    | const   | 8    | 100.00   |
| 1  | department | eq_ref | PRIMARY          | PRIMARY | dept_no | 1    | 100.00   |
+----+------------+--------+------------------+---------+---------+------+----------+</pre><p>
          For <code class="literal">employee</code>, the access method on the
          <code class="literal">name</code> index picks up the 8 rows that match a
          name of <code class="literal">'John'</code>. No filtering is done
          (<code class="literal">filtered</code> is 100%), so all rows are prefix
          rows for the next table: The prefix row count is
          <code class="literal">rows</code> × <code class="literal">filtered</code> =
          8 × 100% = 8.
        </p><p>
          With condition filtering, the optimizer additionally takes
          into account conditions from the <code class="literal">WHERE</code>
          clause not taken into account by the access method. In this
          case, the optimizer uses heuristics to estimate a filtering
          effect of 16.31% for the <a class="link" href="functions.html#operator_between"><code class="literal">BETWEEN</code></a>
          condition on <code class="literal">employee.hire_date</code>. As a
          result, <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> produces output
          like this:
        </p><pre data-lang="none" class="programlisting">+----+------------+--------+------------------+---------+---------+------+----------+
| id | table      | type   | possible_keys    | key     | ref     | rows | filtered |
+----+------------+--------+------------------+---------+---------+------+----------+
| 1  | employee   | ref    | name,h_date,dept | name    | const   | 8    | 16.31    |
| 1  | department | eq_ref | PRIMARY          | PRIMARY | dept_no | 1    | 100.00   |
+----+------------+--------+------------------+---------+---------+------+----------+</pre><p>
          Now the prefix row count is <code class="literal">rows</code> ×
          <code class="literal">filtered</code> = 8 × 16.31% = 1.3, which
          more closely reflects actual data set.
        </p><p>
          Normally, the optimizer does not calculate the condition
          filtering effect (prefix row count reduction) for the last
          joined table because there is no next table to pass rows to.
          An exception occurs for
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>: To provide more
          information, the filtering effect is calculated for all joined
          tables, including the last one.
        </p><p>
          To control whether the optimizer considers additional
          filtering conditions, use the
          <code class="literal">condition_fanout_filter</code> flag of the
          <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
          variable (see <a class="xref" href="optimization.html#switchable-optimizations" title="8.9.2 Switchable Optimizations">Section 8.9.2, “Switchable Optimizations”</a>).
          This flag is enabled by default but can be disabled to
          suppress condition filtering (for example, if a particular
          query is found to yield better performance without it).
        </p><p>
          If the optimizer overestimates the effect of condition
          filtering, performance may be worse than if condition
          filtering is not used. In such cases, these techniques may
          help:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              If a column is not indexed, index it so that the optimizer
              has some information about the distribution of column
              values and can improve its row estimates.
            </p></li><li class="listitem"><p>
              Similarly, if no column histogram information is
              available, generate a histogram (see
              <a class="xref" href="optimization.html#optimizer-statistics" title="8.9.6 Optimizer Statistics">Section 8.9.6, “Optimizer Statistics”</a>).
            </p></li><li class="listitem"><p>
              Change the join order. Ways to accomplish this include
              join-order optimizer hints (see
              <a class="xref" href="optimization.html#optimizer-hints" title="8.9.3 Optimizer Hints">Section 8.9.3, “Optimizer Hints”</a>),
              <code class="literal">STRAIGHT_JOIN</code> immediately following the
              <code class="literal">SELECT</code>, and the
              <code class="literal">STRAIGHT_JOIN</code> join operator.
            </p></li><li class="listitem"><p>
              Disable condition filtering for the session:
            </p><pre data-lang="sql" class="programlisting">SET optimizer_switch = 'condition_fanout_filter=off';</pre><p>
              Or, for a given query, using an optimizer hint:
</p><pre data-lang="sql" class="programlisting">SELECT /*+ SET_VAR(optimizer_switch = 'condition_fanout_filter=off') */ ...</pre></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="constant-folding-optimization"></a>8.2.1.14 Constant-Folding Optimization</h4>

</div>

</div>

</div>
<p>
          Comparisons between constants and column values in which the
          constant value is out of range or of the wrong type with
          respect to the column type are now handled once during query
          optimization rather row-by-row than during execution. The
          comparisons that can be treated in this manner are
          <code class="literal">&gt;</code>, <code class="literal">&gt;=</code>,
          <code class="literal">&lt;</code>, <code class="literal">&lt;=</code>,
          <code class="literal">&lt;&gt;</code>/<code class="literal">!=</code>,
          <code class="literal">=</code>, and <code class="literal">&lt;=&gt;</code>.
        </p><p>
          Consider the table created by the following statement:
        </p><pre data-lang="sql" class="programlisting">CREATE TABLE t (c TINYINT UNSIGNED NOT NULL);</pre><p>
          The <code class="literal">WHERE</code> condition in the query
          <code class="literal">SELECT * FROM t WHERE c &lt; 256</code> contains
          the integral constant 256 which is out of range for a
          <code class="literal">TINYINT UNSIGNED</code> column. Previously, this
          was handled by treating both operands as the larger type, but
          now, since any allowed value for <code class="literal">c</code> is less
          than the constant, the <code class="literal">WHERE</code> expression can
          instead be folded as <code class="literal">WHERE 1</code>, so that the
          query is rewritten as <code class="literal">SELECT * FROM t WHERE
          1</code>.
        </p><p>
          This makes it possible for the optimizer to remove the
          <code class="literal">WHERE</code> expression altogether. If the column
          <code class="literal">c</code> were nullable (that is, defined only as
          <code class="literal">TINYINT UNSIGNED</code>) the query would be
          rewritten like this:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM t WHERE ti IS NOT NULL</pre><p>
          Folding is performed for constants compared to supported MySQL
          column types as follows:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><b>Integer column type. </b>
                Integer types are compared with constants of the
                following types as described here:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p><b>Integer value. </b>
                    If the constant is out of range for the column type,
                    the comparison is folded to <code class="literal">1</code> or
                    <code class="literal">IS NOT NULL</code>, as already shown.
                  </p><p>
                  If the constant is a range boundary, the comparison is
                  folded to <code class="literal">=</code>. For example (using the
                  same table as already defined):
                </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN SELECT * FROM t WHERE c &gt;= 255;</code></strong>
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: t
   partitions: NULL
         type: ALL
possible_keys: NULL
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 5
     filtered: 20.00
        Extra: Using where
1 row in set, 1 warning (0.00 sec)

mysql&gt; <strong class="userinput"><code>SHOW WARNINGS;</code></strong>
*************************** 1. row ***************************
  Level: Note
   Code: 1003
Message: /* select#1 */ select `test`.`t`.`ti` AS `ti` from `test`.`t` where (`test`.`t`.`ti` = 255)
1 row in set (0.00 sec)
</pre></li><li class="listitem"><p><b>Floating- or fixed-point value. </b>
                    If the constant is one of the decimal types (such as
                    <code class="literal">DECIMAL</code>, <code class="literal">REAL</code>,
                    <code class="literal">DOUBLE</code>, or
                    <code class="literal">FLOAT</code>) and has a nonzero decimal
                    portion, it cannot be equal; fold accordingly. For
                    other comparisons, round up or down to an integer
                    value according to the sign, then perform a range
                    check and handle as already described for
                    integer-integer comparisons.
                  </p><p>
                  A <code class="literal">REAL</code> value that is too small to
                  be represented as <code class="literal">DECIMAL</code> is
                  rounded to .01 or -.01 depending on the sign, then
                  handled as a <code class="literal">DECIMAL</code>.
                </p></li><li class="listitem"><p><b>String types. </b>
                    Try to interpret the string value as an integer
                    type, then handle the comparison as between integer
                    values. If this fails, attempt to handle the value
                    as a <code class="literal">REAL</code>.
</p></li></ul>
</div>
</li><li class="listitem"><p><b>DECIMAL or REAL column. </b>
                Decimal types are compared with constants of the
                following types as described here:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p><b>Integer value. </b>
                    Perform a range check against the column
                    value's integer part. If no folding results,
                    convert the constant to <code class="literal">DECIMAL</code>
                    with the same number of decimal places as the column
                    value, then check it as a <code class="literal">DECIMAL</code>
                    (see next).
                  </p></li><li class="listitem"><p><b>DECIMAL or REAL value. </b>
                    Check for overflow (that is, whether the constant
                    has more digits in its integer part than allowed for
                    the column's decimal type). If so, fold.
                  </p><p>
                  If the constant has more significant fractional digits
                  than column's type, truncate the constant. If the
                  comparison operator is <code class="literal">=</code> or
                  <code class="literal">&lt;&gt;</code>, fold. If the operator is
                  <code class="literal">&gt;=</code> or <code class="literal">&lt;=</code>,
                  adjust the operator due to truncation. For example, if
                  column's type is <code class="literal">DECIMAL(3,1)</code>,
                  <code class="literal">SELECT * FROM t WHERE f &gt;= 10.13</code>
                  becomes <code class="literal">SELECT * FROM t WHERE f &gt;
                  10.1</code>.
                </p><p>
                  If the constant has fewer decimal digits than the
                  column's type, convert it to a constant with same
                  number of digits. For underflow of a
                  <code class="literal">REAL</code> value (that is, too few
                  fractional digits to represent it), convert the
                  constant to decimal 0.
                </p></li><li class="listitem"><p><b>String value. </b>
                    If the value can be interpreted as an integer type,
                    handle it as such. Otherwise, try to handle it as
                    <code class="literal">REAL</code>.
</p></li></ul>
</div>
</li><li class="listitem"><p><b>FLOAT or DOUBLE column. </b>
                <code class="literal">FLOAT(<em class="replaceable"><code>m</code></em>,<em class="replaceable"><code>n</code></em>)</code>
                or
                <code class="literal">DOUBLE(<em class="replaceable"><code>m</code></em>,<em class="replaceable"><code>n</code></em>)</code>
                values compared with constants are handled as follows:
              </p><p>
              If the value overflows the range of the column, fold.
            </p><p>
              If the value has more than <em class="replaceable"><code>n</code></em>
              decimals, truncate, compensating during folding. For
              <code class="literal">=</code> and <code class="literal">&lt;&gt;</code>
              comparisons, fold to <code class="literal">TRUE</code>,
              <code class="literal">FALSE</code>, or <code class="literal">IS [NOT]
              NULL</code> as described previously; for other
              operators, adjust the operator.
            </p><p>
              If the value has more than <code class="literal">m</code> integer
              digits, fold.
</p></li></ul>
</div>
<p><b>Limitations. </b>
            This optimization cannot be used in the following cases:
</p>
<div class="orderedlist">
<ol class="orderedlist" type="1"><li class="listitem"><p>
              With comparisons using <code class="literal">BETWEEN</code> or
              <code class="literal">IN</code>.
            </p></li><li class="listitem"><p>
              With <code class="literal">BIT</code> columns or columns using date
              or time types.
            </p></li><li class="listitem"><p>
              During the preparation phase for a prepared statement,
              although it can be applied during the optimization phase
              when the prepared statement is actually executed. This due
              to the fact that, during statement preparation, the value
              of the constant is not yet known.
</p></li></ol>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="is-null-optimization"></a>8.2.1.15 IS NULL Optimization</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444352352576"></a><a class="indexterm" name="idm46444352351504"></a><p>
          MySQL can perform the same optimization on
          <em class="replaceable"><code>col_name</code></em> <a class="link" href="functions.html#operator_is-null"><code class="literal">IS
          NULL</code></a> that it can use for
          <em class="replaceable"><code>col_name</code></em> <code class="literal">=</code>
          <em class="replaceable"><code>constant_value</code></em>. For example, MySQL
          can use indexes and ranges to search for
          <code class="literal">NULL</code> with <a class="link" href="functions.html#operator_is-null"><code class="literal">IS
          NULL</code></a>.
        </p><p>
          Examples:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM <em class="replaceable"><code>tbl_name</code></em> WHERE <em class="replaceable"><code>key_col</code></em> IS NULL;

SELECT * FROM <em class="replaceable"><code>tbl_name</code></em> WHERE <em class="replaceable"><code>key_col</code></em> &lt;=&gt; NULL;

SELECT * FROM <em class="replaceable"><code>tbl_name</code></em>
  WHERE <em class="replaceable"><code>key_col</code></em>=<em class="replaceable"><code>const1</code></em> OR <em class="replaceable"><code>key_col</code></em>=<em class="replaceable"><code>const2</code></em> OR <em class="replaceable"><code>key_col</code></em> IS NULL;
</pre><p>
          If a <code class="literal">WHERE</code> clause includes a
          <em class="replaceable"><code>col_name</code></em> <a class="link" href="functions.html#operator_is-null"><code class="literal">IS
          NULL</code></a> condition for a column that is declared as
          <code class="literal">NOT NULL</code>, that expression is optimized
          away. This optimization does not occur in cases when the
          column might produce <code class="literal">NULL</code> anyway (for
          example, if it comes from a table on the right side of a
          <code class="literal">LEFT JOIN</code>).
        </p><p>
          MySQL can also optimize the combination
          <code class="literal"><em class="replaceable"><code>col_name</code></em> =
          <em class="replaceable"><code>expr</code></em> OR
          <em class="replaceable"><code>col_name</code></em> IS NULL</code>, a form
          that is common in resolved subqueries.
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> shows
          <a class="link" href="optimization.html#jointype_ref_or_null"><code class="literal">ref_or_null</code></a> when this
          optimization is used.
        </p><p>
          This optimization can handle one <a class="link" href="functions.html#operator_is-null"><code class="literal">IS
          NULL</code></a> for any key part.
        </p><p>
          Some examples of queries that are optimized, assuming that
          there is an index on columns <code class="literal">a</code> and
          <code class="literal">b</code> of table <code class="literal">t2</code>:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 WHERE t1.a=<em class="replaceable"><code>expr</code></em> OR t1.a IS NULL;

SELECT * FROM t1, t2 WHERE t1.a=t2.a OR t2.a IS NULL;

SELECT * FROM t1, t2
  WHERE (t1.a=t2.a OR t2.a IS NULL) AND t2.b=t1.b;

SELECT * FROM t1, t2
  WHERE t1.a=t2.a AND (t2.b=t1.b OR t2.b IS NULL);

SELECT * FROM t1, t2
  WHERE (t1.a=t2.a AND t2.a IS NULL AND ...)
  OR (t1.a=t2.a AND t2.a IS NULL AND ...);
</pre><p>
          <a class="link" href="optimization.html#jointype_ref_or_null"><code class="literal">ref_or_null</code></a> works by first
          doing a read on the reference key, and then a separate search
          for rows with a <code class="literal">NULL</code> key value.
        </p><p>
          The optimization can handle only one <a class="link" href="functions.html#operator_is-null"><code class="literal">IS
          NULL</code></a> level. In the following query, MySQL uses key
          lookups only on the expression <code class="literal">(t1.a=t2.a AND t2.a IS
          NULL)</code> and is not able to use the key part on
          <code class="literal">b</code>:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1, t2
  WHERE (t1.a=t2.a AND t2.a IS NULL)
OR (t1.b=t2.b AND t2.b IS NULL);</pre>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="order-by-optimization"></a>8.2.1.16 ORDER BY Optimization</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444352314976"></a><a class="indexterm" name="idm46444352313904"></a><p>
          This section describes when MySQL can use an index to satisfy
          an <code class="literal">ORDER BY</code> clause, the
          <code class="literal">filesort</code> operation used when an index
          cannot be used, and execution plan information available from
          the optimizer about <code class="literal">ORDER BY</code>.
        </p><p>
          An <code class="literal">ORDER BY</code> with and without
          <code class="literal">LIMIT</code> may return rows in different orders,
          as discussed in <a class="xref" href="optimization.html#limit-optimization" title="8.2.1.19 LIMIT Query Optimization">Section 8.2.1.19, “LIMIT Query Optimization”</a>.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a class="xref" href="optimization.html#order-by-index-use" title="Use of Indexes to Satisfy ORDER BY">Use of Indexes to Satisfy ORDER BY</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#order-by-filesort" title="Use of filesort to Satisfy ORDER BY">Use of filesort to Satisfy ORDER BY</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#order-by-optimizer-control" title="Influencing ORDER BY Optimization">Influencing ORDER BY Optimization</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#order-by-diagnostic-information" title="ORDER BY Execution Plan Information Available">ORDER BY Execution Plan Information Available</a></p></li></ul>
</div>

<div class="simplesect">

<div class="titlepage">
<div>

<div class="simple">
<h5 class="title"><a name="order-by-index-use"></a>Use of Indexes to Satisfy ORDER BY</h5>

</div>

</div>

</div>
<p>
            In some cases, MySQL may use an index to satisfy an
            <code class="literal">ORDER BY</code> clause and avoid the extra
            sorting involved in performing a <code class="literal">filesort</code>
            operation.
          </p><p>
            The index may also be used even if the <code class="literal">ORDER
            BY</code> does not match the index exactly, as long as
            all unused portions of the index and all extra
            <code class="literal">ORDER BY</code> columns are constants in the
            <code class="literal">WHERE</code> clause. If the index does not
            contain all columns accessed by the query, the index is used
            only if index access is cheaper than other access methods.
          </p><p>
            Assuming that there is an index on
            <code class="literal">(<em class="replaceable"><code>key_part1</code></em>,
            <em class="replaceable"><code>key_part2</code></em>)</code>, the
            following queries may use the index to resolve the
            <code class="literal">ORDER BY</code> part. Whether the optimizer
            actually does so depends on whether reading the index is
            more efficient than a table scan if columns not in the index
            must also be read.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                In this query, the index on
                <code class="literal">(<em class="replaceable"><code>key_part1</code></em>,
                <em class="replaceable"><code>key_part2</code></em>)</code> enables
                the optimizer to avoid sorting:
              </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1
  ORDER BY <em class="replaceable"><code>key_part1</code></em>, <em class="replaceable"><code>key_part2</code></em>;
</pre><p>
                However, the query uses <code class="literal">SELECT *</code>,
                which may select more columns than
                <em class="replaceable"><code>key_part1</code></em> and
                <em class="replaceable"><code>key_part2</code></em>. In that case,
                scanning an entire index and looking up table rows to
                find columns not in the index may be more expensive than
                scanning the table and sorting the results. If so, the
                optimizer probably will not use the index. If
                <code class="literal">SELECT *</code> selects only the index
                columns, the index will be used and sorting avoided.
              </p><p>
                If <code class="literal">t1</code> is an <code class="literal">InnoDB</code>
                table, the table primary key is implicitly part of the
                index, and the index can be used to resolve the
                <code class="literal">ORDER BY</code> for this query:
              </p><pre data-lang="sql" class="programlisting">SELECT <em class="replaceable"><code>pk</code></em>, <em class="replaceable"><code>key_part1</code></em>, <em class="replaceable"><code>key_part2</code></em> FROM t1
  ORDER BY <em class="replaceable"><code>key_part1</code></em>, <em class="replaceable"><code>key_part2</code></em>;
</pre></li><li class="listitem"><p>
                In this query, <em class="replaceable"><code>key_part1</code></em> is
                constant, so all rows accessed through the index are in
                <em class="replaceable"><code>key_part2</code></em> order, and an index
                on <code class="literal">(<em class="replaceable"><code>key_part1</code></em>,
                <em class="replaceable"><code>key_part2</code></em>)</code> avoids
                sorting if the <code class="literal">WHERE</code> clause is
                selective enough to make an index range scan cheaper
                than a table scan:
              </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1
  WHERE <em class="replaceable"><code>key_part1</code></em> = <em class="replaceable"><code>constant</code></em>
  ORDER BY <em class="replaceable"><code>key_part2</code></em>;
</pre></li><li class="listitem"><p>
                In the next two queries, whether the index is used is
                similar to the same queries without
                <code class="literal">DESC</code> shown previously:
              </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1
  ORDER BY <em class="replaceable"><code>key_part1</code></em> DESC, <em class="replaceable"><code>key_part2</code></em> DESC;

SELECT * FROM t1
  WHERE <em class="replaceable"><code>key_part1</code></em> = <em class="replaceable"><code>constant</code></em>
  ORDER BY <em class="replaceable"><code>key_part2</code></em> DESC;
</pre></li><li class="listitem"><p>
                Two columns in an <code class="literal">ORDER BY</code> can sort
                in the same direction (both <code class="literal">ASC</code>, or
                both <code class="literal">DESC</code>) or in opposite directions
                (one <code class="literal">ASC</code>, one
                <code class="literal">DESC</code>). A condition for index use is
                that the index must have the same homogeneity, but need
                not have the same actual direction.
              </p><p>
                If a query mixes <code class="literal">ASC</code> and
                <code class="literal">DESC</code>, the optimizer can use an index
                on the columns if the index also uses corresponding
                mixed ascending and descending columns:
              </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1
  ORDER BY <em class="replaceable"><code>key_part1</code></em> DESC, <em class="replaceable"><code>key_part2</code></em> ASC;
</pre><p>
                The optimizer can use an index on
                (<em class="replaceable"><code>key_part1</code></em>,
                <em class="replaceable"><code>key_part2</code></em>) if
                <em class="replaceable"><code>key_part1</code></em> is descending and
                <em class="replaceable"><code>key_part2</code></em> is ascending. It
                can also use an index on those columns (with a backward
                scan) if <em class="replaceable"><code>key_part1</code></em> is
                ascending and <em class="replaceable"><code>key_part2</code></em> is
                descending. See <a class="xref" href="optimization.html#descending-indexes" title="8.3.13 Descending Indexes">Section 8.3.13, “Descending Indexes”</a>
              </p></li><li class="listitem"><p>
                In the next two queries,
                <em class="replaceable"><code>key_part1</code></em> is compared to a
                constant. The index will be used if the
                <code class="literal">WHERE</code> clause is selective enough to
                make an index range scan cheaper than a table scan:
              </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1
  WHERE <em class="replaceable"><code>key_part1</code></em> &gt; <em class="replaceable"><code>constant</code></em>
  ORDER BY <em class="replaceable"><code>key_part1</code></em> ASC;

SELECT * FROM t1
  WHERE <em class="replaceable"><code>key_part1</code></em> &lt; <em class="replaceable"><code>constant</code></em>
  ORDER BY <em class="replaceable"><code>key_part1</code></em> DESC;
</pre></li><li class="listitem"><p>
                In the next query, the <code class="literal">ORDER BY</code> does
                not name <em class="replaceable"><code>key_part1</code></em>, but all
                rows selected have a constant
                <em class="replaceable"><code>key_part1</code></em> value, so the index
                can still be used:
              </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1
  WHERE <em class="replaceable"><code>key_part1</code></em> = <em class="replaceable"><code>constant1</code></em> AND <em class="replaceable"><code>key_part2</code></em> &gt; <em class="replaceable"><code>constant2</code></em>
  ORDER BY <em class="replaceable"><code>key_part2</code></em>;
</pre></li></ul>
</div>
<p>
            In some cases, MySQL <span class="emphasis"><em>cannot</em></span> use indexes
            to resolve the <code class="literal">ORDER BY</code>, although it may
            still use indexes to find the rows that match the
            <code class="literal">WHERE</code> clause. Examples:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                The query uses <code class="literal">ORDER BY</code> on different
                indexes:
              </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 ORDER BY <em class="replaceable"><code>key1</code></em>, <em class="replaceable"><code>key2</code></em>;
</pre></li><li class="listitem"><p>
                The query uses <code class="literal">ORDER BY</code> on
                nonconsecutive parts of an index:
              </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 WHERE <em class="replaceable"><code>key2</code></em>=<em class="replaceable"><code>constant</code></em> ORDER BY <em class="replaceable"><code>key1_part1</code></em>, <em class="replaceable"><code>key1_part3</code></em>;
</pre></li><li class="listitem"><p>
                The index used to fetch the rows differs from the one
                used in the <code class="literal">ORDER BY</code>:
              </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 WHERE <em class="replaceable"><code>key2</code></em>=<em class="replaceable"><code>constant</code></em> ORDER BY <em class="replaceable"><code>key1</code></em>;
</pre></li><li class="listitem"><p>
                The query uses <code class="literal">ORDER BY</code> with an
                expression that includes terms other than the index
                column name:
              </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 ORDER BY ABS(<em class="replaceable"><code>key</code></em>);
SELECT * FROM t1 ORDER BY -<em class="replaceable"><code>key</code></em>;
</pre></li><li class="listitem"><p>
                The query joins many tables, and the columns in the
                <code class="literal">ORDER BY</code> are not all from the first
                nonconstant table that is used to retrieve rows. (This
                is the first table in the
                <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output that does
                not have a <a class="link" href="optimization.html#jointype_const"><code class="literal">const</code></a> join
                type.)
              </p></li><li class="listitem"><p>
                The query has different <code class="literal">ORDER BY</code> and
                <code class="literal">GROUP BY</code> expressions.
              </p></li><li class="listitem"><p>
                There is an index on only a prefix of a column named in
                the <code class="literal">ORDER BY</code> clause. In this case,
                the index cannot be used to fully resolve the sort
                order. For example, if only the first 10 bytes of a
                <a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">CHAR(20)</code></a> column are
                indexed, the index cannot distinguish values past the
                10th byte and a <code class="literal">filesort</code> is needed.
              </p></li><li class="listitem"><p>
                The index does not store rows in order. For example,
                this is true for a <code class="literal">HASH</code> index in a
                <code class="literal">MEMORY</code> table.
</p></li></ul>
</div>
<p>
            Availability of an index for sorting may be affected by the
            use of column aliases. Suppose that the column
            <code class="literal">t1.a</code> is indexed. In this statement, the
            name of the column in the select list is
            <code class="literal">a</code>. It refers to <code class="literal">t1.a</code>,
            as does the reference to <code class="literal">a</code> in the
            <code class="literal">ORDER BY</code>, so the index on
            <code class="literal">t1.a</code> can be used:
          </p><pre data-lang="sql" class="programlisting">SELECT a FROM t1 ORDER BY a;</pre><p>
            In this statement, the name of the column in the select list
            is also <code class="literal">a</code>, but it is the alias name. It
            refers to <code class="literal">ABS(a)</code>, as does the reference
            to <code class="literal">a</code> in the <code class="literal">ORDER BY</code>,
            so the index on <code class="literal">t1.a</code> cannot be used:
          </p><pre data-lang="sql" class="programlisting">SELECT ABS(a) AS a FROM t1 ORDER BY a;</pre><p>
            In the following statement, the <code class="literal">ORDER BY</code>
            refers to a name that is not the name of a column in the
            select list. But there is a column in <code class="literal">t1</code>
            named <code class="literal">a</code>, so the <code class="literal">ORDER
            BY</code> refers to <code class="literal">t1.a</code> and the index
            on <code class="literal">t1.a</code> can be used. (The resulting sort
            order may be completely different from the order for
            <code class="literal">ABS(a)</code>, of course.)
          </p><pre data-lang="sql" class="programlisting">SELECT ABS(a) AS b FROM t1 ORDER BY a;</pre><a class="indexterm" name="idm46444352196816"></a><a class="indexterm" name="idm46444352195328"></a><a class="indexterm" name="idm46444352193840"></a><a class="indexterm" name="idm46444352192352"></a><p>
            Previously (MySQL 5.7 and lower),
            <code class="literal">GROUP BY</code> sorted implicitly under certain
            conditions. In MySQL 8.0, that no longer
            occurs, so specifying <code class="literal">ORDER BY NULL</code> at
            the end to suppress implicit sorting (as was done
            previously) is no longer necessary. However, query results
            may differ from previous MySQL versions. To produce a given
            sort order, provide an <code class="literal">ORDER BY</code> clause.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h5 class="title"><a name="order-by-filesort"></a>Use of filesort to Satisfy ORDER BY</h5>

</div>

</div>

</div>
<a class="indexterm" name="idm46444352187024"></a><a class="indexterm" name="idm46444352185984"></a><p>
            If an index cannot be used to satisfy an <code class="literal">ORDER
            BY</code> clause, MySQL performs a
            <code class="literal">filesort</code> operation that reads table rows
            and sorts them. A <code class="literal">filesort</code> constitutes an
            extra sorting phase in query execution.
          </p><p>
            To obtain memory for <code class="literal">filesort</code> operations,
            as of MySQL 8.0.12, the optimizer allocates memory buffers
            incrementally as needed, up to the size indicated by the
            <a class="link" href="server-administration.html#sysvar_sort_buffer_size"><code class="literal">sort_buffer_size</code></a> system
            variable, rather than allocating a fixed amount of
            <a class="link" href="server-administration.html#sysvar_sort_buffer_size"><code class="literal">sort_buffer_size</code></a> bytes up
            front, as was done prior to MySQL 8.0.12. This enables users
            to set <a class="link" href="server-administration.html#sysvar_sort_buffer_size"><code class="literal">sort_buffer_size</code></a> to
            larger values to speed up larger sorts, without concern for
            excessive memory use for small sorts. (This benefit may not
            occur for multiple concurrent sorts on Windows, which has a
            weak multithreaded <code class="literal">malloc</code>.)
          </p><p>
            A <code class="literal">filesort</code> operation uses temporary disk
            files as necessary if the result set is too large to fit in
            memory. Some types of queries are particularly suited to
            completely in-memory <code class="literal">filesort</code> operations.
            For example, the optimizer can use
            <code class="literal">filesort</code> to efficiently handle in memory,
            without temporary files, the <code class="literal">ORDER BY</code>
            operation for queries (and subqueries) of the following
            form:
          </p><pre data-lang="sql" class="programlisting">SELECT ... FROM <em class="replaceable"><code>single_table</code></em> ... ORDER BY <em class="replaceable"><code>non_index_column</code></em> [DESC] LIMIT [<em class="replaceable"><code>M</code></em>,]<em class="replaceable"><code>N</code></em>;
</pre><p>
            Such queries are common in web applications that display
            only a few rows from a larger result set. Examples:
          </p><pre data-lang="sql" class="programlisting">SELECT col1, ... FROM t1 ... ORDER BY name LIMIT 10;
SELECT col1, ... FROM t1 ... ORDER BY RAND() LIMIT 15;</pre>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h5 class="title"><a name="order-by-optimizer-control"></a>Influencing ORDER BY Optimization</h5>

</div>

</div>

</div>
<p>
            For slow <code class="literal">ORDER BY</code> queries for which
            <code class="literal">filesort</code> is not used, try lowering the
            <a class="link" href="server-administration.html#sysvar_max_length_for_sort_data"><code class="literal">max_length_for_sort_data</code></a>
            system variable to a value that is appropriate to trigger a
            <code class="literal">filesort</code>. (A symptom of setting the value
            of this variable too high is a combination of high disk
            activity and low CPU activity.) This technique applies only
            before MySQL 8.0.20. As of 8.0.20,
            <a class="link" href="server-administration.html#sysvar_max_length_for_sort_data"><code class="literal">max_length_for_sort_data</code></a> is
            deprecated due to optimizer changes that make it obsolete
            and of no effect.
          </p><p>
            To increase <code class="literal">ORDER BY</code> speed, check whether
            you can get MySQL to use indexes rather than an extra
            sorting phase. If this is not possible, try the following
            strategies:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                Increase the
                <a class="link" href="server-administration.html#sysvar_sort_buffer_size"><code class="literal">sort_buffer_size</code></a>
                variable value. Ideally, the value should be large
                enough for the entire result set to fit in the sort
                buffer (to avoid writes to disk and merge passes).
              </p><p>
                Take into account that the size of column values stored
                in the sort buffer is affected by the
                <a class="link" href="server-administration.html#sysvar_max_sort_length"><code class="literal">max_sort_length</code></a> system
                variable value. For example, if tuples store values of
                long string columns and you increase the value of
                <a class="link" href="server-administration.html#sysvar_max_sort_length"><code class="literal">max_sort_length</code></a>, the
                size of sort buffer tuples increases as well and may
                require you to increase
                <a class="link" href="server-administration.html#sysvar_sort_buffer_size"><code class="literal">sort_buffer_size</code></a>.
              </p><p>
                To monitor the number of merge passes (to merge
                temporary files), check the
                <a class="link" href="server-administration.html#statvar_Sort_merge_passes"><code class="literal">Sort_merge_passes</code></a>
                status variable.
              </p></li><li class="listitem"><p>
                Increase the
                <a class="link" href="server-administration.html#sysvar_read_rnd_buffer_size"><code class="literal">read_rnd_buffer_size</code></a>
                variable value so that more rows are read at a time.
              </p></li><li class="listitem"><p>
                Change the <a class="link" href="server-administration.html#sysvar_tmpdir"><code class="literal">tmpdir</code></a>
                system variable to point to a dedicated file system with
                large amounts of free space. The variable value can list
                several paths that are used in round-robin fashion; you
                can use this feature to spread the load across several
                directories. Separate the paths by colon characters
                (<code class="literal">:</code>) on Unix and semicolon characters
                (<code class="literal">;</code>) on Windows. The paths should name
                directories in file systems located on different
                <span class="emphasis"><em>physical</em></span> disks, not different
                partitions on the same disk.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h5 class="title"><a name="order-by-diagnostic-information"></a>ORDER BY Execution Plan Information Available</h5>

</div>

</div>

</div>
<p>
            With
            <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>
            (see <a class="xref" href="optimization.html#using-explain" title="8.8.1 Optimizing Queries with EXPLAIN">Section 8.8.1, “Optimizing Queries with EXPLAIN”</a>), you can check whether
            MySQL can use indexes to resolve an <code class="literal">ORDER
            BY</code> clause:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                If the <code class="literal">Extra</code> column of
                <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output does not
                contain <code class="literal">Using filesort</code>, the index is
                used and a <code class="literal">filesort</code> is not performed.
              </p></li><li class="listitem"><p>
                If the <code class="literal">Extra</code> column of
                <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output contains
                <code class="literal">Using filesort</code>, the index is not used
                and a <code class="literal">filesort</code> is performed.
</p></li></ul>
</div>
<p>
            In addition, if a <code class="literal">filesort</code> is performed,
            optimizer trace output includes a
            <code class="literal">filesort_summary</code> block. For example:
          </p><pre data-lang="json" class="programlisting">"filesort_summary": {
  "rows": 100,
  "examined_rows": 100,
  "number_of_tmp_files": 0,
  "peak_memory_used": 25192,
  "sort_mode": "&lt;sort_key, packed_additional_fields&gt;"
}</pre><p>
            <code class="literal">peak_memory_used</code> indicates the maximum
            memory used at any one time during the sort. This is a value
            up to but not necessarily as large as the value of the
            <a class="link" href="server-administration.html#sysvar_sort_buffer_size"><code class="literal">sort_buffer_size</code></a> system
            variable. Prior to MySQL 8.0.12, the output shows
            <code class="literal">sort_buffer_size</code> instead, indicating the
            value of <a class="link" href="server-administration.html#sysvar_sort_buffer_size"><code class="literal">sort_buffer_size</code></a>.
            (Prior to MySQL 8.0.12, the optimizer always allocates
            <a class="link" href="server-administration.html#sysvar_sort_buffer_size"><code class="literal">sort_buffer_size</code></a> bytes for
            the sort buffer. As of 8.0.12, the optimizer allocates
            sort-buffer memory incrementally, beginning with a small
            amount and adding more as necessary, up to
            <a class="link" href="server-administration.html#sysvar_sort_buffer_size"><code class="literal">sort_buffer_size</code></a> bytes.)
          </p><p>
            The <code class="literal">sort_mode</code> value provides information
            about the contents of tuples in the sort buffer:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                <code class="literal">&lt;sort_key, rowid&gt;</code>: This
                indicates that sort buffer tuples are pairs that contain
                the sort key value and row ID of the original table row.
                Tuples are sorted by sort key value and the row ID is
                used to read the row from the table.
              </p></li><li class="listitem"><p>
                <code class="literal">&lt;sort_key, additional_fields&gt;</code>:
                This indicates that sort buffer tuples contain the sort
                key value and columns referenced by the query. Tuples
                are sorted by sort key value and column values are read
                directly from the tuple.
              </p></li><li class="listitem"><p>
                <code class="literal">&lt;sort_key,
                packed_additional_fields&gt;</code>: Like the
                previous variant, but the additional columns are packed
                tightly together instead of using a fixed-length
                encoding.
</p></li></ul>
</div>
<p>
            <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> does not distinguish
            whether the optimizer does or does not perform a
            <code class="literal">filesort</code> in memory. Use of an in-memory
            <code class="literal">filesort</code> can be seen in optimizer trace
            output. Look for
            <code class="literal">filesort_priority_queue_optimization</code>. For
            information about the optimizer trace, see
            <a class="ulink" href="https://dev.mysql.com/doc/internals/en/optimizer-tracing.html" target="_top">MySQL
            Internals: Tracing the Optimizer</a>.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="group-by-optimization"></a>8.2.1.17 GROUP BY Optimization</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444352107104"></a><a class="indexterm" name="idm46444352106032"></a><p>
          The most general way to satisfy a <code class="literal">GROUP BY</code>
          clause is to scan the whole table and create a new temporary
          table where all rows from each group are consecutive, and then
          use this temporary table to discover groups and apply
          aggregate functions (if any). In some cases, MySQL is able to
          do much better than that and avoid creation of temporary
          tables by using index access.
        </p><p>
          The most important preconditions for using indexes for
          <code class="literal">GROUP BY</code> are that all <code class="literal">GROUP
          BY</code> columns reference attributes from the same index,
          and that the index stores its keys in order (as is true, for
          example, for a <code class="literal">BTREE</code> index, but not for a
          <code class="literal">HASH</code> index). Whether use of temporary
          tables can be replaced by index access also depends on which
          parts of an index are used in a query, the conditions
          specified for these parts, and the selected aggregate
          functions.
        </p><p>
          There are two ways to execute a <code class="literal">GROUP BY</code>
          query through index access, as detailed in the following
          sections. The first method applies the grouping operation
          together with all range predicates (if any). The second method
          first performs a range scan, and then groups the resulting
          tuples.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a class="xref" href="optimization.html#loose-index-scan" title="Loose Index Scan">Loose Index Scan</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#tight-index-scan" title="Tight Index Scan">Tight Index Scan</a></p></li></ul>
</div>
<p>
          Loose Index Scan can also be used in the absence of
          <code class="literal">GROUP BY</code> under some conditions. See
          <a class="xref" href="optimization.html#range-access-skip-scan" title="Skip Scan Range Access Method">Skip Scan Range Access Method</a>.
</p>
<div class="simplesect">

<div class="titlepage">
<div>

<div class="simple">
<h5 class="title"><a name="loose-index-scan"></a>Loose Index Scan</h5>
</div>
</div>
</div>
<a class="indexterm" name="idm46444352092464"></a><p>
            The most efficient way to process <code class="literal">GROUP
            BY</code> is when an index is used to directly retrieve
            the grouping columns. With this access method, MySQL uses
            the property of some index types that the keys are ordered
            (for example, <code class="literal">BTREE</code>). This property
            enables use of lookup groups in an index without having to
            consider all keys in the index that satisfy all
            <code class="literal">WHERE</code> conditions. This access method
            considers only a fraction of the keys in an index, so it is
            called a <span class="firstterm">Loose Index
            Scan</span>. When there is no <code class="literal">WHERE</code>
            clause, a Loose Index Scan reads as many keys as the number
            of groups, which may be a much smaller number than that of
            all keys. If the <code class="literal">WHERE</code> clause contains
            range predicates (see the discussion of the
            <a class="link" href="optimization.html#jointype_range"><code class="literal">range</code></a> join type in
            <a class="xref" href="optimization.html#using-explain" title="8.8.1 Optimizing Queries with EXPLAIN">Section 8.8.1, “Optimizing Queries with EXPLAIN”</a>), a Loose Index Scan looks
            up the first key of each group that satisfies the range
            conditions, and again reads the smallest possible number of
            keys. This is possible under the following conditions:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                The query is over a single table.
              </p></li><li class="listitem"><p>
                The <code class="literal">GROUP BY</code> names only columns that
                form a leftmost prefix of the index and no other
                columns. (If, instead of <code class="literal">GROUP BY</code>,
                the query has a <code class="literal">DISTINCT</code> clause, all
                distinct attributes refer to columns that form a
                leftmost prefix of the index.) For example, if a table
                <code class="literal">t1</code> has an index on
                <code class="literal">(c1,c2,c3)</code>, Loose Index Scan is
                applicable if the query has <code class="literal">GROUP BY c1,
                c2</code>. It is not applicable if the query has
                <code class="literal">GROUP BY c2, c3</code> (the columns are not
                a leftmost prefix) or <code class="literal">GROUP BY c1, c2,
                c4</code> (<code class="literal">c4</code> is not in the
                index).
              </p></li><li class="listitem"><p>
                The only aggregate functions used in the select list (if
                any) are <a class="link" href="functions.html#function_min"><code class="literal">MIN()</code></a> and
                <a class="link" href="functions.html#function_max"><code class="literal">MAX()</code></a>, and all of them
                refer to the same column. The column must be in the
                index and must immediately follow the columns in the
                <code class="literal">GROUP BY</code>.
              </p></li><li class="listitem"><p>
                Any other parts of the index than those from the
                <code class="literal">GROUP BY</code> referenced in the query must
                be constants (that is, they must be referenced in
                equalities with constants), except for the argument of
                <a class="link" href="functions.html#function_min"><code class="literal">MIN()</code></a> or
                <a class="link" href="functions.html#function_max"><code class="literal">MAX()</code></a> functions.
              </p></li><li class="listitem"><p>
                For columns in the index, full column values must be
                indexed, not just a prefix. For example, with
                <code class="literal">c1 VARCHAR(20), INDEX (c1(10))</code>, the
                index uses only a prefix of <code class="literal">c1</code> values
                and cannot be used for Loose Index Scan.
</p></li></ul>
</div>
<p>
            If Loose Index Scan is applicable to a query, the
            <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output shows
            <code class="literal">Using index for group-by</code> in the
            <code class="literal">Extra</code> column.
          </p><p>
            Assume that there is an index
            <code class="literal">idx(c1,c2,c3)</code> on table
            <code class="literal">t1(c1,c2,c3,c4)</code>. The Loose Index Scan
            access method can be used for the following queries:
          </p><pre data-lang="sql" class="programlisting">SELECT c1, c2 FROM t1 GROUP BY c1, c2;
SELECT DISTINCT c1, c2 FROM t1;
SELECT c1, MIN(c2) FROM t1 GROUP BY c1;
SELECT c1, c2 FROM t1 WHERE c1 &lt; <em class="replaceable"><code>const</code></em> GROUP BY c1, c2;
SELECT MAX(c3), MIN(c3), c1, c2 FROM t1 WHERE c2 &gt; <em class="replaceable"><code>const</code></em> GROUP BY c1, c2;
SELECT c2 FROM t1 WHERE c1 &lt; <em class="replaceable"><code>const</code></em> GROUP BY c1, c2;
SELECT c1, c2 FROM t1 WHERE c3 = <em class="replaceable"><code>const</code></em> GROUP BY c1, c2;
</pre><p>
            The following queries cannot be executed with this quick
            select method, for the reasons given:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                There are aggregate functions other than
                <a class="link" href="functions.html#function_min"><code class="literal">MIN()</code></a> or
                <a class="link" href="functions.html#function_max"><code class="literal">MAX()</code></a>:
              </p><pre data-lang="sql" class="programlisting">SELECT c1, SUM(c2) FROM t1 GROUP BY c1;</pre></li><li class="listitem"><p>
                The columns in the <code class="literal">GROUP BY</code> clause do
                not form a leftmost prefix of the index:
              </p><pre data-lang="sql" class="programlisting">SELECT c1, c2 FROM t1 GROUP BY c2, c3;</pre></li><li class="listitem"><p>
                The query refers to a part of a key that comes after the
                <code class="literal">GROUP BY</code> part, and for which there is
                no equality with a constant:
              </p><pre data-lang="sql" class="programlisting">SELECT c1, c3 FROM t1 GROUP BY c1, c2;</pre><p>
                Were the query to include <code class="literal">WHERE c3 =
                <em class="replaceable"><code>const</code></em></code>, Loose Index
                Scan could be used.
</p></li></ul>
</div>
<p>
            The Loose Index Scan access method can be applied to other
            forms of aggregate function references in the select list,
            in addition to the <a class="link" href="functions.html#function_min"><code class="literal">MIN()</code></a> and
            <a class="link" href="functions.html#function_max"><code class="literal">MAX()</code></a> references already
            supported:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                <a class="link" href="functions.html#function_avg"><code class="literal">AVG(DISTINCT)</code></a>,
                <a class="link" href="functions.html#function_sum"><code class="literal">SUM(DISTINCT)</code></a>, and
                <a class="link" href="functions.html#function_count"><code class="literal">COUNT(DISTINCT)</code></a> are
                supported. <a class="link" href="functions.html#function_avg"><code class="literal">AVG(DISTINCT)</code></a>
                and <a class="link" href="functions.html#function_sum"><code class="literal">SUM(DISTINCT)</code></a> take a
                single argument.
                <a class="link" href="functions.html#function_count"><code class="literal">COUNT(DISTINCT)</code></a> can have
                more than one column argument.
              </p></li><li class="listitem"><p>
                There must be no <code class="literal">GROUP BY</code> or
                <code class="literal">DISTINCT</code> clause in the query.
              </p></li><li class="listitem"><p>
                The Loose Index Scan limitations described previously
                still apply.
</p></li></ul>
</div>
<p>
            Assume that there is an index
            <code class="literal">idx(c1,c2,c3)</code> on table
            <code class="literal">t1(c1,c2,c3,c4)</code>. The Loose Index Scan
            access method can be used for the following queries:
          </p><pre data-lang="sql" class="programlisting">SELECT COUNT(DISTINCT c1), SUM(DISTINCT c1) FROM t1;

SELECT COUNT(DISTINCT c1, c2), COUNT(DISTINCT c2, c1) FROM t1;</pre>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h5 class="title"><a name="tight-index-scan"></a>Tight Index Scan</h5>

</div>

</div>

</div>
<p>
            A Tight Index Scan may be either a full index scan or a
            range index scan, depending on the query conditions.
          </p><p>
            When the conditions for a Loose Index Scan are not met, it
            still may be possible to avoid creation of temporary tables
            for <code class="literal">GROUP BY</code> queries. If there are range
            conditions in the <code class="literal">WHERE</code> clause, this
            method reads only the keys that satisfy these conditions.
            Otherwise, it performs an index scan. Because this method
            reads all keys in each range defined by the
            <code class="literal">WHERE</code> clause, or scans the whole index if
            there are no range conditions, it is called a
            <span class="firstterm">Tight Index Scan</span>. With
            a Tight Index Scan, the grouping operation is performed only
            after all keys that satisfy the range conditions have been
            found.
          </p><p>
            For this method to work, it is sufficient that there be a
            constant equality condition for all columns in a query
            referring to parts of the key coming before or in between
            parts of the <code class="literal">GROUP BY</code> key. The constants
            from the equality conditions fill in any <span class="quote">“<span class="quote">gaps</span>”</span>
            in the search keys so that it is possible to form complete
            prefixes of the index. These index prefixes then can be used
            for index lookups. If the <code class="literal">GROUP BY</code> result
            requires sorting, and it is possible to form search keys
            that are prefixes of the index, MySQL also avoids extra
            sorting operations because searching with prefixes in an
            ordered index already retrieves all the keys in order.
          </p><p>
            Assume that there is an index
            <code class="literal">idx(c1,c2,c3)</code> on table
            <code class="literal">t1(c1,c2,c3,c4)</code>. The following queries do
            not work with the Loose Index Scan access method described
            previously, but still work with the Tight Index Scan access
            method.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
                There is a gap in the <code class="literal">GROUP BY</code>, but
                it is covered by the condition <code class="literal">c2 =
                'a'</code>:
              </p><pre data-lang="sql" class="programlisting">SELECT c1, c2, c3 FROM t1 WHERE c2 = 'a' GROUP BY c1, c3;</pre></li><li class="listitem"><p>
                The <code class="literal">GROUP BY</code> does not begin with the
                first part of the key, but there is a condition that
                provides a constant for that part:
</p><pre data-lang="sql" class="programlisting">SELECT c1, c2, c3 FROM t1 WHERE c1 = 'a' GROUP BY c2, c3;</pre></li></ul>
</div>

</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="distinct-optimization"></a>8.2.1.18 DISTINCT Optimization</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444352006880"></a><a class="indexterm" name="idm46444352005808"></a><p>
          <code class="literal">DISTINCT</code> combined with <code class="literal">ORDER
          BY</code> needs a temporary table in many cases.
        </p><p>
          Because <code class="literal">DISTINCT</code> may use <code class="literal">GROUP
          BY</code>, learn how MySQL works with columns in
          <code class="literal">ORDER BY</code> or <code class="literal">HAVING</code>
          clauses that are not part of the selected columns. See
          <a class="xref" href="functions.html#group-by-handling" title="12.20.3 MySQL Handling of GROUP BY">Section 12.20.3, “MySQL Handling of GROUP BY”</a>.
        </p><p>
          In most cases, a <code class="literal">DISTINCT</code> clause can be
          considered as a special case of <code class="literal">GROUP BY</code>.
          For example, the following two queries are equivalent:
        </p><pre data-lang="sql" class="programlisting">SELECT DISTINCT c1, c2, c3 FROM t1
WHERE c1 &gt; <em class="replaceable"><code>const</code></em>;

SELECT c1, c2, c3 FROM t1
WHERE c1 &gt; <em class="replaceable"><code>const</code></em> GROUP BY c1, c2, c3;
</pre><p>
          Due to this equivalence, the optimizations applicable to
          <code class="literal">GROUP BY</code> queries can be also applied to
          queries with a <code class="literal">DISTINCT</code> clause. Thus, for
          more details on the optimization possibilities for
          <code class="literal">DISTINCT</code> queries, see
          <a class="xref" href="optimization.html#group-by-optimization" title="8.2.1.17 GROUP BY Optimization">Section 8.2.1.17, “GROUP BY Optimization”</a>.
        </p><p>
          When combining <code class="literal">LIMIT
          <em class="replaceable"><code>row_count</code></em></code> with
          <code class="literal">DISTINCT</code>, MySQL stops as soon as it finds
          <em class="replaceable"><code>row_count</code></em> unique rows.
        </p><p>
          If you do not use columns from all tables named in a query,
          MySQL stops scanning any unused tables as soon as it finds the
          first match. In the following case, assuming that
          <code class="literal">t1</code> is used before <code class="literal">t2</code>
          (which you can check with
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>), MySQL stops reading
          from <code class="literal">t2</code> (for any particular row in
          <code class="literal">t1</code>) when it finds the first row in
          <code class="literal">t2</code>:
</p><pre data-lang="sql" class="programlisting">SELECT DISTINCT t1.a FROM t1, t2 where t1.a=t2.a;</pre>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="limit-optimization"></a>8.2.1.19 LIMIT Query Optimization</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444351980736"></a><a class="indexterm" name="idm46444351979280"></a><p>
          If you need only a specified number of rows from a result set,
          use a <code class="literal">LIMIT</code> clause in the query, rather
          than fetching the whole result set and throwing away the extra
          data.
        </p><p>
          MySQL sometimes optimizes a query that has a <code class="literal">LIMIT
          <em class="replaceable"><code>row_count</code></em></code> clause and no
          <code class="literal">HAVING</code> clause:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              If you select only a few rows with
              <code class="literal">LIMIT</code>, MySQL uses indexes in some cases
              when normally it would prefer to do a full table scan.
            </p></li><li class="listitem"><p>
              If you combine <code class="literal">LIMIT
              <em class="replaceable"><code>row_count</code></em></code> with
              <code class="literal">ORDER BY</code>, MySQL stops sorting as soon
              as it has found the first
              <em class="replaceable"><code>row_count</code></em> rows of the sorted
              result, rather than sorting the entire result. If ordering
              is done by using an index, this is very fast. If a
              filesort must be done, all rows that match the query
              without the <code class="literal">LIMIT</code> clause are selected,
              and most or all of them are sorted, before the first
              <em class="replaceable"><code>row_count</code></em> are found. After the
              initial rows have been found, MySQL does not sort any
              remainder of the result set.
            </p><p>
              One manifestation of this behavior is that an
              <code class="literal">ORDER BY</code> query with and without
              <code class="literal">LIMIT</code> may return rows in different
              order, as described later in this section.
            </p></li><li class="listitem"><p>
              If you combine <code class="literal">LIMIT
              <em class="replaceable"><code>row_count</code></em></code> with
              <code class="literal">DISTINCT</code>, MySQL stops as soon as it
              finds <em class="replaceable"><code>row_count</code></em> unique rows.
            </p></li><li class="listitem"><p>
              In some cases, a <code class="literal">GROUP BY</code> can be
              resolved by reading the index in order (or doing a sort on
              the index), then calculating summaries until the index
              value changes. In this case, <code class="literal">LIMIT
              <em class="replaceable"><code>row_count</code></em></code> does not
              calculate any unnecessary <code class="literal">GROUP BY</code>
              values.
            </p></li><li class="listitem"><p>
              As soon as MySQL has sent the required number of rows to
              the client, it aborts the query unless you are using
              <code class="literal">SQL_CALC_FOUND_ROWS</code>. In that case, the
              number of rows can be retrieved with <code class="literal">SELECT
              FOUND_ROWS()</code>. See
              <a class="xref" href="functions.html#information-functions" title="12.15 Information Functions">Section 12.15, “Information Functions”</a>.
            </p><a class="indexterm" name="idm46444351956128"></a></li><li class="listitem"><p>
              <code class="literal">LIMIT 0</code> quickly returns an empty set.
              This can be useful for checking the validity of a query.
              It can also be employed to obtain the types of the result
              columns within applications that use a MySQL API that
              makes result set metadata available. With the
              <a class="link" href="programs.html#mysql" title="4.5.1 mysql — The MySQL Command-Line Client"><span class="command"><strong>mysql</strong></span></a> client program, you can use the
              <a class="link" href="programs.html#option_mysql_column-type-info"><code class="option">--column-type-info</code></a> option to
              display result column types.
            </p></li><li class="listitem"><p>
              If the server uses temporary tables to resolve a query, it
              uses the <code class="literal">LIMIT
              <em class="replaceable"><code>row_count</code></em></code> clause to
              calculate how much space is required.
            </p></li><li class="listitem"><p>
              If an index is not used for <code class="literal">ORDER BY</code>
              but a <code class="literal">LIMIT</code> clause is also present, the
              optimizer may be able to avoid using a merge file and sort
              the rows in memory using an in-memory
              <code class="literal">filesort</code> operation.
</p></li></ul>
</div>
<p>
          If multiple rows have identical values in the <code class="literal">ORDER
          BY</code> columns, the server is free to return those rows
          in any order, and may do so differently depending on the
          overall execution plan. In other words, the sort order of
          those rows is nondeterministic with respect to the nonordered
          columns.
        </p><p>
          One factor that affects the execution plan is
          <code class="literal">LIMIT</code>, so an <code class="literal">ORDER BY</code>
          query with and without <code class="literal">LIMIT</code> may return
          rows in different orders. Consider this query, which is sorted
          by the <code class="literal">category</code> column but nondeterministic
          with respect to the <code class="literal">id</code> and
          <code class="literal">rating</code> columns:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT * FROM ratings ORDER BY category;</code></strong>
+----+----------+--------+
| id | category | rating |
+----+----------+--------+
|  1 |        1 |    4.5 |
|  5 |        1 |    3.2 |
|  3 |        2 |    3.7 |
|  4 |        2 |    3.5 |
|  6 |        2 |    3.5 |
|  2 |        3 |    5.0 |
|  7 |        3 |    2.7 |
+----+----------+--------+
</pre><p>
          Including <code class="literal">LIMIT</code> may affect order of rows
          within each <code class="literal">category</code> value. For example,
          this is a valid query result:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT * FROM ratings ORDER BY category LIMIT 5;</code></strong>
+----+----------+--------+
| id | category | rating |
+----+----------+--------+
|  1 |        1 |    4.5 |
|  5 |        1 |    3.2 |
|  4 |        2 |    3.5 |
|  3 |        2 |    3.7 |
|  6 |        2 |    3.5 |
+----+----------+--------+
</pre><p>
          In each case, the rows are sorted by the <code class="literal">ORDER
          BY</code> column, which is all that is required by the SQL
          standard.
        </p><p>
          If it is important to ensure the same row order with and
          without <code class="literal">LIMIT</code>, include additional columns
          in the <code class="literal">ORDER BY</code> clause to make the order
          deterministic. For example, if <code class="literal">id</code> values
          are unique, you can make rows for a given
          <code class="literal">category</code> value appear in
          <code class="literal">id</code> order by sorting like this:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT * FROM ratings ORDER BY category, id;</code></strong>
+----+----------+--------+
| id | category | rating |
+----+----------+--------+
|  1 |        1 |    4.5 |
|  5 |        1 |    3.2 |
|  3 |        2 |    3.7 |
|  4 |        2 |    3.5 |
|  6 |        2 |    3.5 |
|  2 |        3 |    5.0 |
|  7 |        3 |    2.7 |
+----+----------+--------+

mysql&gt; <strong class="userinput"><code>SELECT * FROM ratings ORDER BY category, id LIMIT 5;</code></strong>
+----+----------+--------+
| id | category | rating |
+----+----------+--------+
|  1 |        1 |    4.5 |
|  5 |        1 |    3.2 |
|  3 |        2 |    3.7 |
|  4 |        2 |    3.5 |
|  6 |        2 |    3.5 |
+----+----------+--------+
</pre><p>
          For a query with an <code class="literal">ORDER BY</code> or
          <code class="literal">GROUP BY</code> and a <code class="literal">LIMIT</code>
          clause, the optimizer tries to choose an ordered index by
          default, which is generally faster for small
          <code class="literal">LIMIT</code> values. Prior to MySQL 8.0.21, there
          was no way to override this behavior, even in cases where
          using some other optimization might be faster. Beginning with
          MySQL 8.0.21, it is possible to turn off this optimization by
          setting the <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a>
          system variable's
          <code class="literal">prefer_ordering_index</code> flag to
          <code class="literal">off</code>.
        </p><p>
          You should be aware that, with
          <code class="literal">prefer_ordering_index</code> set to
          <code class="literal">off</code>, some <code class="literal">GROUP BY</code>
          queries using the
          <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">GROUP_INDEX</code></a> optimizer hint
          may not execute successfully if the
          <code class="literal">ONLY_FULL_GROUP_BY</code> server SQL mode is in
          effect (the default). For example:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>CREATE TABLE t (</code></strong>
    -&gt;     <strong class="userinput"><code>id1 BIGINT NOT NULL,</code></strong>
    -&gt;     <strong class="userinput"><code>id2 BIGINT NOT NULL,</code></strong>
    -&gt;     <strong class="userinput"><code>c1 VARCHAR(50) NOT NULL,</code></strong>
    -&gt;     <strong class="userinput"><code>c2 VARCHAR(50) NOT NULL,</code></strong>
    -&gt;  <strong class="userinput"><code>PRIMARY KEY (id1),</code></strong>
    -&gt;  <strong class="userinput"><code>INDEX i (id2, c1)</code></strong>
    -&gt; <strong class="userinput"><code>);</code></strong>

# [Insert some rows into table t]

# Verify that prefer_ordering_index flag is enabled,
# and that SQL mode includes FULL_GROUP_BY

mysql&gt; <strong class="userinput"><code>SELECT @@optimizer_switch LIKE '%prefer_ordering_index=on%';</code></strong>
+------------------------------------------------------+
| @@optimizer_switch LIKE '%prefer_ordering_index=on%' |
+------------------------------------------------------+
|                                                    1 |
+------------------------------------------------------+

mysql&gt; <strong class="userinput"><code>SELECT @@sql_mode LIKE '%ONLY_FULL_GROUP_BY%';</code></strong>
+----------------------------------------+
| @@sql_mode LIKE '%ONLY_FULL_GROUP_BY%' |
+----------------------------------------+
|                                      1 |
+----------------------------------------+
1 row in set (0.00 sec)

# GROUP BY query with GROUP_INDEX

mysql&gt; <strong class="userinput"><code>EXPLAIN /*+ GROUP_INDEX(t i) */ SELECT c2 FROM t</code></strong>
    -&gt;     <strong class="userinput"><code>WHERE id1 &gt; 8</code></strong>
    -&gt;     <strong class="userinput"><code>GROUP BY id2 LIMIT 1\G</code></strong>
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: t
   partitions: NULL
         type: range
possible_keys: PRIMARY,i
          key: PRIMARY
      key_len: 8
          ref: NULL
         rows: 2
     filtered: 100.00
        Extra: Using where; Using temporary

# Disable prefer_ordering_index flag

mysql&gt; <strong class="userinput"><code>SET @@optimizer_switch = "prefer_ordering_index=off";
</code></strong>
# Query now fails
mysql&gt; <strong class="userinput"><code>EXPLAIN /*+ GROUP_INDEX(t i) */ SELECT c2 FROM t</code></strong>
    -&gt;     <strong class="userinput"><code>WHERE id1 &gt; 8</code></strong>
    -&gt;     <strong class="userinput"><code>GROUP BY id2 LIMIT 1\G</code></strong>
<span class="errortext">ERROR 1055 (42000): Expression #1 of SELECT list is not in GROUP BY clause and
contains nonaggregated column 'poi.t.c2' which is not functionally dependent on
columns in GROUP BY clause; this is incompatible with
sql_mode=only_full_group_by</span>

# Remove ONLY_FULL_GROUP_BY from server SQL mode

mysql&gt; <strong class="userinput"><code>SET @@sql_mode = REPLACE(@@sql_mode, 'ONLY_FULL_GROUP_BY,', '');</code></strong>

# Re-run query; it now succeeds

mysql&gt; <strong class="userinput"><code>EXPLAIN /*+ GROUP_INDEX(t i) */ SELECT c2 FROM t</code></strong>
    -&gt;     <strong class="userinput"><code>WHERE id1 &gt; 8</code></strong>
    -&gt;     <strong class="userinput"><code>GROUP BY id2 LIMIT 1\G</code></strong>
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: t
   partitions: NULL
         type: range
possible_keys: PRIMARY,i
          key: PRIMARY
      key_len: 8
          ref: NULL
         rows: 2
     filtered: 100.00
        Extra: Using where; Using temporary
</pre>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="function-optimization"></a>8.2.1.20 Function Call Optimization</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444351892880"></a><a class="indexterm" name="idm46444351891408"></a><a class="indexterm" name="idm46444351889904"></a><a class="indexterm" name="idm46444351888400"></a><p>
          MySQL functions are tagged internally as deterministic or
          nondeterministic. A function is nondeterministic if, given
          fixed values for its arguments, it can return different
          results for different invocations. Examples of
          nondeterministic functions:
          <a class="link" href="functions.html#function_rand"><code class="literal">RAND()</code></a>,
          <a class="link" href="functions.html#function_uuid"><code class="literal">UUID()</code></a>.
        </p><p>
          If a function is tagged nondeterministic, a reference to it in
          a <code class="literal">WHERE</code> clause is evaluated for every row
          (when selecting from one table) or combination of rows (when
          selecting from a multiple-table join).
        </p><p>
          MySQL also determines when to evaluate functions based on
          types of arguments, whether the arguments are table columns or
          constant values. A deterministic function that takes a table
          column as argument must be evaluated whenever that column
          changes value.
        </p><p>
          Nondeterministic functions may affect query performance. For
          example, some optimizations may not be available, or more
          locking might be required. The following discussion uses
          <a class="link" href="functions.html#function_rand"><code class="literal">RAND()</code></a> but applies to other
          nondeterministic functions as well.
        </p><p>
          Suppose that a table <code class="literal">t</code> has this definition:
        </p><pre data-lang="sql" class="programlisting">CREATE TABLE t (id INT NOT NULL PRIMARY KEY, col_a VARCHAR(100));</pre><p>
          Consider these two queries:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM t WHERE id = POW(1,2);
SELECT * FROM t WHERE id = FLOOR(1 + RAND() * 49);</pre><p>
          Both queries appear to use a primary key lookup because of the
          equality comparison against the primary key, but that is true
          only for the first of them:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              The first query always produces a maximum of one row
              because <a class="link" href="functions.html#function_pow"><code class="literal">POW()</code></a> with constant
              arguments is a constant value and is used for index
              lookup.
            </p></li><li class="listitem"><p>
              The second query contains an expression that uses the
              nondeterministic function
              <a class="link" href="functions.html#function_rand"><code class="literal">RAND()</code></a>, which is not
              constant in the query but in fact has a new value for
              every row of table <code class="literal">t</code>. Consequently, the
              query reads every row of the table, evaluates the
              predicate for each row, and outputs all rows for which the
              primary key matches the random value. This might be zero,
              one, or multiple rows, depending on the
              <code class="literal">id</code> column values and the values in the
              <a class="link" href="functions.html#function_rand"><code class="literal">RAND()</code></a> sequence.
</p></li></ul>
</div>
<p>
          The effects of nondeterminism are not limited to
          <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements. This
          <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a> statement uses a
          nondeterministic function to select rows to be modified:
        </p><pre data-lang="sql" class="programlisting">UPDATE t SET col_a = <em class="replaceable"><code>some_expr</code></em> WHERE id = FLOOR(1 + RAND() * 49);
</pre><p>
          Presumably the intent is to update at most a single row for
          which the primary key matches the expression. However, it
          might update zero, one, or multiple rows, depending on the
          <code class="literal">id</code> column values and the values in the
          <a class="link" href="functions.html#function_rand"><code class="literal">RAND()</code></a> sequence.
        </p><p>
          The behavior just described has implications for performance
          and replication:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Because a nondeterministic function does not produce a
              constant value, the optimizer cannot use strategies that
              might otherwise be applicable, such as index lookups. The
              result may be a table scan.
            </p></li><li class="listitem"><p>
              <code class="literal">InnoDB</code> might escalate to a range-key
              lock rather than taking a single row lock for one matching
              row.
            </p></li><li class="listitem"><p>
              Updates that do not execute deterministically are unsafe
              for replication.
</p></li></ul>
</div>
<p>
          The difficulties stem from the fact that the
          <a class="link" href="functions.html#function_rand"><code class="literal">RAND()</code></a> function is evaluated
          once for every row of the table. To avoid multiple function
          evaluations, use one of these techniques:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Move the expression containing the nondeterministic
              function to a separate statement, saving the value in a
              variable. In the original statement, replace the
              expression with a reference to the variable, which the
              optimizer can treat as a constant value:
            </p><pre data-lang="sql" class="programlisting">SET @keyval = FLOOR(1 + RAND() * 49);
UPDATE t SET col_a = <em class="replaceable"><code>some_expr</code></em> WHERE id = @keyval;
</pre></li><li class="listitem"><p>
              Assign the random value to a variable in a derived table.
              This technique causes the variable to be assigned a value,
              once, prior to its use in the comparison in the
              <code class="literal">WHERE</code> clause:
            </p><pre data-lang="sql" class="programlisting">UPDATE /*+ NO_MERGE(dt) */ t, (SELECT FLOOR(1 + RAND() * 49) AS r) AS dt
SET col_a = <em class="replaceable"><code>some_expr</code></em> WHERE id = dt.r;
</pre></li></ul>
</div>
<p>
          As mentioned previously, a nondeterministic expression in the
          <code class="literal">WHERE</code> clause might prevent optimizations
          and result in a table scan. However, it may be possible to
          partially optimize the <code class="literal">WHERE</code> clause if
          other expressions are deterministic. For example:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM t WHERE partial_key=5 AND some_column=RAND();</pre><p>
          If the optimizer can use <code class="literal">partial_key</code> to
          reduce the set of rows selected,
          <a class="link" href="functions.html#function_rand"><code class="literal">RAND()</code></a> is executed fewer times,
          which diminishes the effect of nondeterminism on optimization.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="window-function-optimization"></a>8.2.1.21 Window Function Optimization</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444351841152"></a><a class="indexterm" name="idm46444351839664"></a><p>
          Window functions affect the strategies the optimizer
          considers:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Derived table merging for a subquery is disabled if the
              subquery has window functions. The subquery is always
              materialized.
            </p></li><li class="listitem"><p>
              Semijoins are not applicable to window function
              optimization because semijoins apply to subqueries in
              <code class="literal">WHERE</code> and <code class="literal">JOIN ...
              ON</code>, which cannot contain window functions.
            </p></li><li class="listitem"><p>
              The optimizer processes multiple windows that have the
              same ordering requirements in sequence, so sorting can be
              skipped for windows following the first one.
            </p></li><li class="listitem"><p>
              The optimizer makes no attempt to merge windows that could
              be evaluated in a single step (for example, when multiple
              <code class="literal">OVER</code> clauses contain identical window
              definitions). The workaround is to define the window in a
              <code class="literal">WINDOW</code> clause and refer to the window
              name in the <code class="literal">OVER</code> clauses.
</p></li></ul>
</div>
<p>
          An aggregate function not used as a window function is
          aggregated in the outermost possible query. For example, in
          this query, MySQL sees that <code class="literal">COUNT(t1.b)</code> is
          something that cannot exist in the outer query because of its
          placement in the <code class="literal">WHERE</code> clause:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 WHERE t1.a = (SELECT COUNT(t1.b) FROM t2);</pre><p>
          Consequently, MySQL aggregates inside the subquery, treating
          <code class="literal">t1.b</code> as a constant and returning the count
          of rows of <code class="literal">t2</code>.
        </p><p>
          Replacing <code class="literal">WHERE</code> with
          <code class="literal">HAVING</code> results in an error:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT * FROM t1 HAVING t1.a = (SELECT COUNT(t1.b) FROM t2);</code></strong>
ERROR 1140 (42000): In aggregated query without GROUP BY, expression #1
of SELECT list contains nonaggregated column 'test.t1.a'; this is
incompatible with sql_mode=only_full_group_by
</pre><p>
          The error occurs because <code class="literal">COUNT(t1.b)</code> can
          exist in the <code class="literal">HAVING</code>, and so makes the outer
          query aggregated.
        </p><p>
          Window functions (including aggregate functions used as window
          functions) do not have the preceding complexity. They always
          aggregate in the subquery where they are written, never in the
          outer query.
        </p><p>
          Window function evaluation may be affected by the value of the
          <a class="link" href="server-administration.html#sysvar_windowing_use_high_precision"><code class="literal">windowing_use_high_precision</code></a>
          system variable, which determines whether to compute window
          operations without loss of precision. By default,
          <a class="link" href="server-administration.html#sysvar_windowing_use_high_precision"><code class="literal">windowing_use_high_precision</code></a>
          is enabled.
        </p><p>
          For some moving frame aggregates, the inverse aggregate
          function can be applied to remove values from the aggregate.
          This can improve performance but possibly with a loss of
          precision. For example, adding a very small floating-point
          value to a very large value causes the very small value to be
          <span class="quote">“<span class="quote">hidden</span>”</span> by the large value. When inverting the
          large value later, the effect of the small value is lost.
        </p><p>
          Loss of precision due to inverse aggregation is a factor only
          for operations on floating-point (approximate-value) data
          types. For other types, inverse aggregation is safe; this
          includes <a class="link" href="data-types.html#fixed-point-types" title="11.1.3 Fixed-Point Types (Exact Value) - DECIMAL, NUMERIC"><code class="literal">DECIMAL</code></a>, which permits
          a fractional part but is an exact-value type.
        </p><p>
          For faster execution, MySQL always uses inverse aggregation
          when it is safe:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              For floating-point values, inverse aggregation is not
              always safe and might result in loss of precision. The
              default is to avoid inverse aggregation, which is slower
              but preserves precision. If it is permissible to sacrifice
              safety for speed,
              <a class="link" href="server-administration.html#sysvar_windowing_use_high_precision"><code class="literal">windowing_use_high_precision</code></a>
              can be disabled to permit inverse aggregation.
            </p></li><li class="listitem"><p>
              For nonfloating-point data types, inverse aggregation is
              always safe and is used regardless of the
              <a class="link" href="server-administration.html#sysvar_windowing_use_high_precision"><code class="literal">windowing_use_high_precision</code></a>
              value.
            </p></li><li class="listitem"><p>
              <a class="link" href="server-administration.html#sysvar_windowing_use_high_precision"><code class="literal">windowing_use_high_precision</code></a>
              has no effect on <a class="link" href="functions.html#function_min"><code class="literal">MIN()</code></a> and
              <a class="link" href="functions.html#function_max"><code class="literal">MAX()</code></a>, which do not use
              inverse aggregation in any case.
</p></li></ul>
</div>
<p>
          For evaluation of the variance functions
          <a class="link" href="functions.html#function_stddev-pop"><code class="literal">STDDEV_POP()</code></a>,
          <a class="link" href="functions.html#function_stddev-samp"><code class="literal">STDDEV_SAMP()</code></a>,
          <a class="link" href="functions.html#function_var-pop"><code class="literal">VAR_POP()</code></a>,
          <a class="link" href="functions.html#function_var-samp"><code class="literal">VAR_SAMP()</code></a>, and their synonyms,
          evaluation can occur in optimized mode or default mode.
          Optimized mode may produce slightly different results in the
          last significant digits. If such differences are permissible,
          <a class="link" href="server-administration.html#sysvar_windowing_use_high_precision"><code class="literal">windowing_use_high_precision</code></a>
          can be disabled to permit optimized mode.
        </p><a class="indexterm" name="idm46444351794096"></a><a class="indexterm" name="idm46444351792608"></a><p>
          For <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>, windowing
          execution plan information is too extensive to display in
          traditional output format. To see windowing information, use
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN
          FORMAT=JSON</code></a> and look for the
          <code class="literal">windowing</code> element.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="row-constructor-optimization"></a>8.2.1.22 Row Constructor Expression Optimization</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444351786064"></a><a class="indexterm" name="idm46444351784576"></a><p>
          Row constructors permit simultaneous comparisons of multiple
          values. For example, these two statements are semantically
          equivalent:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 WHERE (column1,column2) = (1,1);
SELECT * FROM t1 WHERE column1 = 1 AND column2 = 1;</pre><p>
          In addition, the optimizer handles both expressions the same
          way.
        </p><p>
          The optimizer is less likely to use available indexes if the
          row constructor columns do not cover the prefix of an index.
          Consider the following table, which has a primary key on
          <code class="literal">(c1, c2, c3)</code>:
        </p><pre data-lang="sql" class="programlisting">CREATE TABLE t1 (
  c1 INT, c2 INT, c3 INT, c4 CHAR(100),
  PRIMARY KEY(c1,c2,c3)
);</pre><p>
          In this query, the <code class="literal">WHERE</code> clause uses all
          columns in the index. However, the row constructor itself does
          not cover an index prefix, with the result that the optimizer
          uses only <code class="literal">c1</code> (<code class="literal">key_len=4</code>,
          the size of <code class="literal">c1</code>):
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN SELECT * FROM t1</code></strong>
       <strong class="userinput"><code>WHERE c1=1 AND (c2,c3) &gt; (1,1)\G</code></strong>
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: t1
   partitions: NULL
         type: ref
possible_keys: PRIMARY
          key: PRIMARY
      key_len: 4
          ref: const
         rows: 3
     filtered: 100.00
        Extra: Using where
</pre><p>
          In such cases, rewriting the row constructor expression using
          an equivalent nonconstructor expression may result in more
          complete index use. For the given query, the row constructor
          and equivalent nonconstructor expressions are:
        </p><pre data-lang="sql" class="programlisting">(c2,c3) &gt; (1,1)
c2 &gt; 1 OR ((c2 = 1) AND (c3 &gt; 1))</pre><p>
          Rewriting the query to use the nonconstructor expression
          results in the optimizer using all three columns in the index
          (<code class="literal">key_len=12</code>):
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN SELECT * FROM t1</code></strong>
       <strong class="userinput"><code>WHERE c1 = 1 AND (c2 &gt; 1 OR ((c2 = 1) AND (c3 &gt; 1)))\G</code></strong>
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: t1
   partitions: NULL
         type: range
possible_keys: PRIMARY
          key: PRIMARY
      key_len: 12
          ref: NULL
         rows: 3
     filtered: 100.00
        Extra: Using where
</pre><p>
          Thus, for better results, avoid mixing row constructors with
          <a class="link" href="functions.html#operator_and"><code class="literal">AND</code></a>/<a class="link" href="functions.html#operator_or"><code class="literal">OR</code></a>
          expressions. Use one or the other.
        </p><p>
          Under certain conditions, the optimizer can apply the range
          access method to <a class="link" href="functions.html#operator_in"><code class="literal">IN()</code></a> expressions
          that have row constructor arguments. See
          <a class="xref" href="optimization.html#row-constructor-range-optimization" title="Range Optimization of Row Constructor Expressions">Range Optimization of Row Constructor Expressions</a>.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="table-scan-avoidance"></a>8.2.1.23 Avoiding Full Table Scans</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444351760144"></a><a class="indexterm" name="idm46444351758688"></a><a class="indexterm" name="idm46444351757200"></a><p>
          The output from <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> shows
          <a class="link" href="optimization.html#jointype_all"><code class="literal">ALL</code></a> in the
          <code class="literal">type</code> column when MySQL uses a
          <a class="link" href="glossary.html#glos_full_table_scan" title="full table scan">full table scan</a> to
          resolve a query. This usually happens under the following
          conditions:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              The table is so small that it is faster to perform a table
              scan than to bother with a key lookup. This is common for
              tables with fewer than 10 rows and a short row length.
            </p></li><li class="listitem"><p>
              There are no usable restrictions in the
              <code class="literal">ON</code> or <code class="literal">WHERE</code> clause
              for indexed columns.
            </p></li><li class="listitem"><p>
              You are comparing indexed columns with constant values and
              MySQL has calculated (based on the index tree) that the
              constants cover too large a part of the table and that a
              table scan would be faster. See
              <a class="xref" href="optimization.html#where-optimization" title="8.2.1.1 WHERE Clause Optimization">Section 8.2.1.1, “WHERE Clause Optimization”</a>.
            </p></li><li class="listitem"><p>
              You are using a key with low cardinality (many rows match
              the key value) through another column. In this case, MySQL
              assumes that by using the key it probably will do many key
              lookups and that a table scan would be faster.
</p></li></ul>
</div>
<p>
          For small tables, a table scan often is appropriate and the
          performance impact is negligible. For large tables, try the
          following techniques to avoid having the optimizer incorrectly
          choose a table scan:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Use <code class="literal">ANALYZE TABLE
              <em class="replaceable"><code>tbl_name</code></em></code> to update
              the key distributions for the scanned table. See
              <a class="xref" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement">Section 13.7.3.1, “ANALYZE TABLE Statement”</a>.
            </p></li><li class="listitem"><p>
              Use <code class="literal">FORCE INDEX</code> for the scanned table
              to tell MySQL that table scans are very expensive compared
              to using the given index:
            </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1, t2 FORCE INDEX (<em class="replaceable"><code>index_for_column</code></em>)
  WHERE t1.<em class="replaceable"><code>col_name</code></em>=t2.<em class="replaceable"><code>col_name</code></em>;
</pre><p>
              See <a class="xref" href="optimization.html#index-hints" title="8.9.4 Index Hints">Section 8.9.4, “Index Hints”</a>.
            </p></li><li class="listitem"><p>
              Start <a class="link" href="programs.html#mysqld" title="4.3.1 mysqld — The MySQL Server"><span class="command"><strong>mysqld</strong></span></a> with the
              <a class="link" href="server-administration.html#sysvar_max_seeks_for_key"><code class="option">--max-seeks-for-key=1000</code></a>
              option or use <code class="literal">SET
              max_seeks_for_key=1000</code> to tell the optimizer to
              assume that no key scan causes more than 1,000 key seeks.
              See <a class="xref" href="server-administration.html#server-system-variables" title="5.1.8 Server System Variables">Section 5.1.8, “Server System Variables”</a>.
</p></li></ul>
</div>

</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="subquery-optimization"></a>8.2.2 Optimizing Subqueries, Derived Tables, View References, and Common Table
Expressions</h3>
</div>
</div>
</div>
<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#semijoins">8.2.2.1 Optimizing IN and EXISTS Subquery Predicates with Semijoin
          Transformations</a></span></dt><dt><span class="section"><a href="optimization.html#subquery-materialization">8.2.2.2 Optimizing Subqueries with Materialization</a></span></dt><dt><span class="section"><a href="optimization.html#subquery-optimization-with-exists">8.2.2.3 Optimizing Subqueries with the EXISTS Strategy</a></span></dt><dt><span class="section"><a href="optimization.html#derived-table-optimization">8.2.2.4 Optimizing Derived Tables, View References, and Common Table Expressions
with Merging or Materialization</a></span></dt></dl>
</div>
<a class="indexterm" name="idm46444351730352"></a><a class="indexterm" name="idm46444351728896"></a><a class="indexterm" name="idm46444351727408"></a><a class="indexterm" name="idm46444351725920"></a><a class="indexterm" name="idm46444351724432"></a><a class="indexterm" name="idm46444351722944"></a><a class="indexterm" name="idm46444351721456"></a><a class="indexterm" name="idm46444351719952"></a><p>
        The MySQL query optimizer has different strategies available to
        evaluate subqueries:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            For a subquery used with an <code class="literal">IN</code>,
            <code class="literal">= ANY</code>, or <code class="literal">EXISTS</code>
            predicate, the optimizer has these choices:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                Semijoin
              </p></li><li class="listitem"><p>
                Materialization
              </p></li><li class="listitem"><p>
                <code class="literal">EXISTS</code> strategy
</p></li></ul>
</div>
</li><li class="listitem"><p>
            For a subquery used with a <code class="literal">NOT IN</code>,
            <code class="literal">&lt;&gt; ALL</code> or <code class="literal">NOT
            EXISTS</code> predicate, the optimizer has these choices:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                Materialization
              </p></li><li class="listitem"><p>
                <code class="literal">EXISTS</code> strategy
</p></li></ul>
</div>
</li></ul>
</div>
<p>
        For a derived table, the optimizer has these choices (which also
        apply to view references and common table expressions):
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Merge the derived table into the outer query block
          </p></li><li class="listitem"><p>
            Materialize the derived table to an internal temporary table
</p></li></ul>
</div>
<p>
        The following discussion provides more information about the
        preceding optimization strategies.
</p>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
          A limitation on <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a> and
          <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a> statements that use a
          subquery to modify a single table is that the optimizer does
          not use semijoin or materialization subquery optimizations. As
          a workaround, try rewriting them as multiple-table
          <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a> and
          <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a> statements that use a
          join rather than a subquery.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="semijoins"></a>8.2.2.1 Optimizing IN and EXISTS Subquery Predicates with Semijoin
Transformations</h4>
</div>
</div>
</div>
<a class="indexterm" name="idm46444351694112"></a><p>
          A semijoin is a preparation-time transformation that enables
          multiple execution strategies such as table pullout, duplicate
          weedout, first match, loose scan, and materialization. The
          optimizer uses semijoin strategies to improve subquery
          execution, as described in this section.
        </p><p>
          For an inner join between two tables, the join returns a row
          from one table as many times as there are matches in the other
          table. But for some questions, the only information that
          matters is whether there is a match, not the number of
          matches. Suppose that there are tables named
          <code class="literal">class</code> and <code class="literal">roster</code> that
          list classes in a course curriculum and class rosters
          (students enrolled in each class), respectively. To list the
          classes that actually have students enrolled, you could use
          this join:
        </p><pre data-lang="sql" class="programlisting">SELECT class.class_num, class.class_name
    FROM class
    INNER JOIN roster
    WHERE class.class_num = roster.class_num;</pre><p>
          However, the result lists each class once for each enrolled
          student. For the question being asked, this is unnecessary
          duplication of information.
        </p><p>
          Assuming that <code class="literal">class_num</code> is a primary key in
          the <code class="literal">class</code> table, duplicate suppression is
          possible by using
          <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT
          DISTINCT</code></a>, but it is inefficient to generate all
          matching rows first only to eliminate duplicates later.
        </p><p>
          The same duplicate-free result can be obtained by using a
          subquery:
        </p><pre data-lang="sql" class="programlisting">SELECT class_num, class_name
    FROM class
    WHERE class_num IN
        (SELECT class_num FROM roster);</pre><p>
          Here, the optimizer can recognize that the
          <code class="literal">IN</code> clause requires the subquery to return
          only one instance of each class number from the
          <code class="literal">roster</code> table. In this case, the query can
          use a <span class="firstterm">semijoin</span>; that is,
          an operation that returns only one instance of each row in
          <code class="literal">class</code> that is matched by rows in
          <code class="literal">roster</code>.
        </p><p>
          The following statement, which contains an
          <code class="literal">EXISTS</code> subquery predicate, is equivalent to
          the previous statement containing an <code class="literal">IN</code>
          subquery predicate:
        </p><pre data-lang="sql" class="programlisting">SELECT class_num, class_name
    FROM class
    WHERE EXISTS
        (SELECT * FROM roster WHERE class.class_num = roster.class_num);</pre><p>
          In MySQL 8.0.16 and later, any statement with an
          <code class="literal">EXISTS</code> subquery predicate is subject to the
          same semijoin transforms as a statement with an equivalent
          <code class="literal">IN</code> subquery predicate.
        </p><p>
          Beginning with MySQL 8.0.17, the following subqueries are
          transformed into antijoins:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <code class="literal">NOT IN (SELECT ... FROM ...)</code>
            </p></li><li class="listitem"><p>
              <code class="literal">NOT EXISTS (SELECT ... FROM ...)</code>.
            </p></li><li class="listitem"><p>
              <code class="literal">IN (SELECT ... FROM ...) IS NOT TRUE</code>
            </p></li><li class="listitem"><p>
              <code class="literal">EXISTS (SELECT ... FROM ...) IS NOT
              TRUE</code>.
            </p></li><li class="listitem"><p>
              <code class="literal">IN (SELECT ... FROM ...) IS FALSE</code>
            </p></li><li class="listitem"><p>
              <code class="literal">EXISTS (SELECT ... FROM ...) IS FALSE</code>.
</p></li></ul>
</div>
<p>
          In short, any negation of a subquery of the form <code class="literal">IN
          (SELECT ... FROM ...)</code> or <code class="literal">EXISTS (SELECT ...
          FROM ...)</code> is transformed into an antijoin.
        </p><p>
          An antijoin is an operation that returns only rows for which
          there is no match. Consider the query shown here:
        </p><pre data-lang="sql" class="programlisting">SELECT class_num, class_name
    FROM class
    WHERE class_num NOT IN
        (SELECT class_num FROM roster);</pre><p>
          This query is rewritten internally as the antijoin
          <code class="literal">SELECT class_num, class_name FROM class ANTIJOIN
          roster ON class_num</code>, which returns one instance of
          each row in <code class="literal">class</code> that is
          <span class="emphasis"><em>not</em></span> matched by any rows in
          <code class="literal">roster</code>. This means that, for each row in
          <code class="literal">class</code>, as soon as a match is found in
          <code class="literal">roster</code>, the row in <code class="literal">class</code>
          can be discarded.
        </p><p>
          Antijoin transformations cannot in most cases be applied if
          the expressions being compared are nullable. An exception to
          this rule is that <code class="literal">(... NOT IN (SELECT ...)) IS NOT
          FALSE</code> and its equivalent <code class="literal">(... IN (SELECT
          ...)) IS NOT TRUE</code> can be transformed into antijoins.
        </p><p>
          Outer join and inner join syntax is permitted in the outer
          query specification, and table references may be base tables,
          derived tables, view references, or common table expressions.
        </p><p>
          In MySQL, a subquery must satisfy these criteria to be handled
          as a semijoin (or, in MySQL 8.0.17 and later, an antijoin if
          <code class="literal">NOT</code> modifies the subquery):
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              It must be part of an <code class="literal">IN</code>, <code class="literal">=
              ANY</code>, or <code class="literal">EXISTS</code> predicate that
              appears at the top level of the <code class="literal">WHERE</code>
              or <code class="literal">ON</code> clause, possibly as a term in an
              <code class="literal">AND</code> expression. For example:
            </p><pre data-lang="sql" class="programlisting">SELECT ...
    FROM ot1, ...
    WHERE (oe1, ...) IN
        (SELECT ie1, ... FROM it1, ... WHERE ...);</pre><p>
              Here, <code class="literal">ot_<em class="replaceable"><code>i</code></em></code>
              and <code class="literal">it_<em class="replaceable"><code>i</code></em></code>
              represent tables in the outer and inner parts of the
              query, and
              <code class="literal">oe_<em class="replaceable"><code>i</code></em></code> and
              <code class="literal">ie_<em class="replaceable"><code>i</code></em></code>
              represent expressions that refer to columns in the outer
              and inner tables.
            </p><p>
              In MySQL 8.0.17 and later, the subquery can also be the
              argument to an expression modified by
              <code class="literal">NOT</code>, <code class="literal">IS [NOT] TRUE</code>,
              or <code class="literal">IS [NOT] FALSE</code>.
            </p></li><li class="listitem"><p>
              It must be a single <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>
              without <a class="link" href="sql-statements.html#union" title="13.2.10.3 UNION Clause"><code class="literal">UNION</code></a> constructs.
            </p></li><li class="listitem"><p>
              It must not contain a <code class="literal">HAVING</code> clause.
            </p></li><li class="listitem"><p>
              It must not contain any aggregate functions (whether it is
              explicitly or implicitly grouped).
            </p></li><li class="listitem"><p>
              It must not have a <code class="literal">LIMIT</code> clause.
            </p></li><li class="listitem"><p>
              The statement must not use the
              <code class="literal">STRAIGHT_JOIN</code> join type in the outer
              query.
            </p><a class="indexterm" name="idm46444351629152"></a></li><li class="listitem"><p>
              The <code class="literal">STRAIGHT_JOIN</code> modifier must not be
              present.
            </p><a class="indexterm" name="idm46444351626096"></a></li><li class="listitem"><p>
              The number of outer and inner tables together must be less
              than the maximum number of tables permitted in a join.
            </p></li><li class="listitem"><p>
              The subquery may be correlated or uncorrelated. In MySQL
              8.0.16 and later, decorrelation looks at trivially
              correlated predicates in the <code class="literal">WHERE</code>
              clause of a subquery used as the argument to
              <code class="literal">EXISTS</code>, and makes it possible to
              optimize it as if it was used within <code class="literal">IN (SELECT b
              FROM ...)</code>. The term <span class="emphasis"><em>trivially
              correlated</em></span> means that the predicate is an
              equality predicate, that it is the sole predicate in the
              <code class="literal">WHERE</code> clause (or is combined with
              <code class="literal">AND</code>), and that one operand is from a
              table referenced in the subquery and the other operand is
              from the outer query block.
            </p></li><li class="listitem"><p>
              The <code class="literal">DISTINCT</code> keyword is permitted but
              ignored. Semijoin strategies automatically handle
              duplicate removal.
            </p></li><li class="listitem"><p>
              A <code class="literal">GROUP BY</code> clause is permitted but
              ignored, unless the subquery also contains one or more
              aggregate functions.
            </p></li><li class="listitem"><p>
              An <code class="literal">ORDER BY</code> clause is permitted but
              ignored, since ordering is irrelevant to the evaluation of
              semijoin strategies.
</p></li></ul>
</div>
<p>
          If a subquery meets the preceding criteria, MySQL converts it
          to a semijoin (or, in MySQL 8.0.17 or later, an antijoin if
          applicable) and makes a cost-based choice from these
          strategies:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Convert the subquery to a join, or use table pullout and
              run the query as an inner join between subquery tables and
              outer tables. Table pullout pulls a table out from the
              subquery to the outer query.
            </p><a class="indexterm" name="idm46444351611248"></a></li><li class="listitem"><p>
              <span class="emphasis"><em>Duplicate Weedout</em></span>: Run the semijoin
              as if it was a join and remove duplicate records using a
              temporary table.
            </p><a class="indexterm" name="idm46444351608432"></a></li><li class="listitem"><p>
              <span class="emphasis"><em>FirstMatch</em></span>: When scanning the inner
              tables for row combinations and there are multiple
              instances of a given value group, choose one rather than
              returning them all. This "shortcuts" scanning and
              eliminates production of unnecessary rows.
            </p><a class="indexterm" name="idm46444351605456"></a></li><li class="listitem"><p>
              <span class="emphasis"><em>LooseScan</em></span>: Scan a subquery table
              using an index that enables a single value to be chosen
              from each subquery's value group.
            </p><a class="indexterm" name="idm46444351602608"></a></li><li class="listitem"><p>
              Materialize the subquery into an indexed temporary table
              that is used to perform a join, where the index is used to
              remove duplicates. The index might also be used later for
              lookups when joining the temporary table with the outer
              tables; if not, the table is scanned. For more information
              about materialization, see
              <a class="xref" href="optimization.html#subquery-materialization" title="8.2.2.2 Optimizing Subqueries with Materialization">Section 8.2.2.2, “Optimizing Subqueries with Materialization”</a>.
</p><a class="indexterm" name="idm46444351598992"></a></li></ul>
</div>
<p>
          Each of these strategies can be enabled or disabled using the
          following <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a>
          system variable flags:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              The <code class="literal">semijoin</code> flag controls whether
              semijoins are used. Starting with MySQL 8.0.17, this also
              applies to antijoins.
            </p></li><li class="listitem"><p>
              If <code class="literal">semijoin</code> is enabled, the
              <code class="literal">firstmatch</code>,
              <code class="literal">loosescan</code>,
              <code class="literal">duplicateweedout</code>, and
              <code class="literal">materialization</code> flags enable finer
              control over the permitted semijoin strategies.
            </p></li><li class="listitem"><p>
              If the <code class="literal">duplicateweedout</code> semijoin
              strategy is disabled, it is not used unless all other
              applicable strategies are also disabled.
            </p></li><li class="listitem"><p>
              If <code class="literal">duplicateweedout</code> is disabled, on
              occasion the optimizer may generate a query plan that is
              far from optimal. This occurs due to heuristic pruning
              during greedy search, which can be avoided by setting
              <a class="link" href="server-administration.html#sysvar_optimizer_prune_level"><code class="literal">optimizer_prune_level=0</code></a>.
</p></li></ul>
</div>
<p>
          These flags are enabled by default. See
          <a class="xref" href="optimization.html#switchable-optimizations" title="8.9.2 Switchable Optimizations">Section 8.9.2, “Switchable Optimizations”</a>.
        </p><p>
          The optimizer minimizes differences in handling of views and
          derived tables. This affects queries that use the
          <code class="literal">STRAIGHT_JOIN</code> modifier and a view with an
          <code class="literal">IN</code> subquery that can be converted to a
          semijoin. The following query illustrates this because the
          change in processing causes a change in transformation, and
          thus a different execution strategy:
        </p><pre data-lang="sql" class="programlisting">CREATE VIEW v AS
SELECT *
FROM t1
WHERE a IN (SELECT b
           FROM t2);

SELECT STRAIGHT_JOIN *
FROM t3 JOIN v ON t3.x = v.a;</pre><p>
          The optimizer first looks at the view and converts the
          <code class="literal">IN</code> subquery to a semijoin, then checks
          whether it is possible to merge the view into the outer query.
          Because the <code class="literal">STRAIGHT_JOIN</code> modifier in the
          outer query prevents semijoin, the optimizer refuses the
          merge, causing derived table evaluation using a materialized
          table.
        </p><p>
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output indicates the
          use of semijoin strategies as follows:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              For extended <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>
              output, the text displayed by a following
              <a class="link" href="sql-statements.html#show-warnings" title="13.7.7.40 SHOW WARNINGS Statement"><code class="literal">SHOW WARNINGS</code></a> shows the
              rewritten query, which displays the semijoin structure.
              (See <a class="xref" href="optimization.html#explain-extended" title="8.8.3 Extended EXPLAIN Output Format">Section 8.8.3, “Extended EXPLAIN Output Format”</a>.) From this you
              can get an idea about which tables were pulled out of the
              semijoin. If a subquery was converted to a semijoin, you
              will see that the subquery predicate is gone and its
              tables and <code class="literal">WHERE</code> clause were merged
              into the outer query join list and
              <code class="literal">WHERE</code> clause.
            </p></li><li class="listitem"><p>
              Temporary table use for Duplicate Weedout is indicated by
              <code class="literal">Start temporary</code> and <code class="literal">End
              temporary</code> in the <code class="literal">Extra</code>
              column. Tables that were not pulled out and are in the
              range of <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output
              rows covered by <code class="literal">Start temporary</code> and
              <code class="literal">End temporary</code> have their
              <code class="literal">rowid</code> in the temporary table.
            </p></li><li class="listitem"><p>
              <code class="literal">FirstMatch(<em class="replaceable"><code>tbl_name</code></em>)</code>
              in the <code class="literal">Extra</code> column indicates join
              shortcutting.
            </p></li><li class="listitem"><p>
              <code class="literal">LooseScan(<em class="replaceable"><code>m</code></em>..<em class="replaceable"><code>n</code></em>)</code>
              in the <code class="literal">Extra</code> column indicates use of
              the LooseScan strategy. <em class="replaceable"><code>m</code></em> and
              <em class="replaceable"><code>n</code></em> are key part numbers.
            </p></li><li class="listitem"><p>
              Temporary table use for materialization is indicated by
              rows with a <code class="literal">select_type</code> value of
              <code class="literal">MATERIALIZED</code> and rows with a
              <code class="literal">table</code> value of
              <code class="literal">&lt;subquery<em class="replaceable"><code>N</code></em>&gt;</code>.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="subquery-materialization"></a>8.2.2.2 Optimizing Subqueries with Materialization</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444351551200"></a><a class="indexterm" name="idm46444351550160"></a><a class="indexterm" name="idm46444351548672"></a><p>
          The optimizer uses materialization to enable more efficient
          subquery processing. Materialization speeds up query execution
          by generating a subquery result as a temporary table, normally
          in memory. The first time MySQL needs the subquery result, it
          materializes that result into a temporary table. Any
          subsequent time the result is needed, MySQL refers again to
          the temporary table. The optimizer may index the table with a
          hash index to make lookups fast and inexpensive. The index
          contains unique values to eliminate duplicates and make the
          table smaller.
        </p><p>
          Subquery materialization uses an in-memory temporary table
          when possible, falling back to on-disk storage if the table
          becomes too large. See
          <a class="xref" href="optimization.html#internal-temporary-tables" title="8.4.4 Internal Temporary Table Use in MySQL">Section 8.4.4, “Internal Temporary Table Use in MySQL”</a>.
        </p><p>
          If materialization is not used, the optimizer sometimes
          rewrites a noncorrelated subquery as a correlated subquery.
          For example, the following <code class="literal">IN</code> subquery is
          noncorrelated (<em class="replaceable"><code>where_condition</code></em>
          involves only columns from <code class="literal">t2</code> and not
          <code class="literal">t1</code>):
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1
WHERE t1.a IN (SELECT t2.b FROM t2 WHERE <em class="replaceable"><code>where_condition</code></em>);
</pre><p>
          The optimizer might rewrite this as an
          <code class="literal">EXISTS</code> correlated subquery:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1
WHERE EXISTS (SELECT t2.b FROM t2 WHERE <em class="replaceable"><code>where_condition</code></em> AND t1.a=t2.b);
</pre><p>
          Subquery materialization using a temporary table avoids such
          rewrites and makes it possible to execute the subquery only
          once rather than once per row of the outer query.
        </p><p>
          For subquery materialization to be used in MySQL, the
          <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
          variable <code class="literal">materialization</code> flag must be
          enabled. (See <a class="xref" href="optimization.html#switchable-optimizations" title="8.9.2 Switchable Optimizations">Section 8.9.2, “Switchable Optimizations”</a>.)
          With the <code class="literal">materialization</code> flag enabled,
          materialization applies to subquery predicates that appear
          anywhere (in the select list, <code class="literal">WHERE</code>,
          <code class="literal">ON</code>, <code class="literal">GROUP BY</code>,
          <code class="literal">HAVING</code>, or <code class="literal">ORDER BY</code>),
          for predicates that fall into any of these use cases:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              The predicate has this form, when no outer expression
              <em class="replaceable"><code>oe_i</code></em> or inner expression
              <em class="replaceable"><code>ie_i</code></em> is nullable.
              <em class="replaceable"><code>N</code></em> is 1 or larger.
            </p><pre data-lang="sql" class="programlisting">(<em class="replaceable"><code>oe_1</code></em>, <em class="replaceable"><code>oe_2</code></em>, ..., <em class="replaceable"><code>oe_N</code></em>) [NOT] IN (SELECT <em class="replaceable"><code>ie_1</code></em>, <em class="replaceable"><code>i_2</code></em>, ..., <em class="replaceable"><code>ie_N</code></em> ...)
</pre></li><li class="listitem"><p>
              The predicate has this form, when there is a single outer
              expression <em class="replaceable"><code>oe</code></em> and inner
              expression <em class="replaceable"><code>ie</code></em>. The expressions
              can be nullable.
            </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>oe</code></em> [NOT] IN (SELECT <em class="replaceable"><code>ie</code></em> ...)
</pre></li><li class="listitem"><p>
              The predicate is <code class="literal">IN</code> or <code class="literal">NOT
              IN</code> and a result of <code class="literal">UNKNOWN</code>
              (<code class="literal">NULL</code>) has the same meaning as a result
              of <code class="literal">FALSE</code>.
</p></li></ul>
</div>
<p>
          The following examples illustrate how the requirement for
          equivalence of <code class="literal">UNKNOWN</code> and
          <code class="literal">FALSE</code> predicate evaluation affects whether
          subquery materialization can be used. Assume that
          <em class="replaceable"><code>where_condition</code></em> involves columns
          only from <code class="literal">t2</code> and not <code class="literal">t1</code>
          so that the subquery is noncorrelated.
        </p><p>
          This query is subject to materialization:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1
WHERE t1.a IN (SELECT t2.b FROM t2 WHERE <em class="replaceable"><code>where_condition</code></em>);
</pre><p>
          Here, it does not matter whether the <code class="literal">IN</code>
          predicate returns <code class="literal">UNKNOWN</code> or
          <code class="literal">FALSE</code>. Either way, the row from
          <code class="literal">t1</code> is not included in the query result.
        </p><p>
          An example where subquery materialization is not used is the
          following query, where <code class="literal">t2.b</code> is a nullable
          column:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1
WHERE (t1.a,t1.b) NOT IN (SELECT t2.a,t2.b FROM t2
                          WHERE <em class="replaceable"><code>where_condition</code></em>);
</pre><p>
          The following restrictions apply to the use of subquery
          materialization:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              The types of the inner and outer expressions must match.
              For example, the optimizer might be able to use
              materialization if both expressions are integer or both
              are decimal, but cannot if one expression is integer and
              the other is decimal.
            </p></li><li class="listitem"><p>
              The inner expression cannot be a
              <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a>.
</p></li></ul>
</div>
<p>
          Use of <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> with a query
          provides some indication of whether the optimizer uses
          subquery materialization:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Compared to query execution that does not use
              materialization, <code class="literal">select_type</code> may change
              from <code class="literal">DEPENDENT SUBQUERY</code> to
              <code class="literal">SUBQUERY</code>. This indicates that, for a
              subquery that would be executed once per outer row,
              materialization enables the subquery to be executed just
              once.
            </p></li><li class="listitem"><p>
              For extended <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>
              output, the text displayed by a following
              <a class="link" href="sql-statements.html#show-warnings" title="13.7.7.40 SHOW WARNINGS Statement"><code class="literal">SHOW WARNINGS</code></a> includes
              <code class="literal">materialize</code> and
              <code class="literal">materialized-subquery</code>.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="subquery-optimization-with-exists"></a>8.2.2.3 Optimizing Subqueries with the EXISTS Strategy</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444351488080"></a><a class="indexterm" name="idm46444351486592"></a><p>
          Certain optimizations are applicable to comparisons that use
          the <code class="literal">IN</code> (or <code class="literal">=ANY</code>)
          operator to test subquery results. This section discusses
          these optimizations, particularly with regard to the
          challenges that <code class="literal">NULL</code> values present. The
          last part of the discussion suggests how you can help the
          optimizer.
        </p><p>
          Consider the following subquery comparison:
        </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>outer_expr</code></em> IN (SELECT <em class="replaceable"><code>inner_expr</code></em> FROM ... WHERE <em class="replaceable"><code>subquery_where</code></em>)
</pre><p>
          MySQL evaluates queries <span class="quote">“<span class="quote">from outside to inside.</span>”</span>
          That is, it first obtains the value of the outer expression
          <em class="replaceable"><code>outer_expr</code></em>, and then runs the
          subquery and captures the rows that it produces.
        </p><p>
          A very useful optimization is to <span class="quote">“<span class="quote">inform</span>”</span> the
          subquery that the only rows of interest are those where the
          inner expression <em class="replaceable"><code>inner_expr</code></em> is
          equal to <em class="replaceable"><code>outer_expr</code></em>. This is done
          by pushing down an appropriate equality into the
          subquery's <code class="literal">WHERE</code> clause to make it
          more restrictive. The converted comparison looks like this:
        </p><pre data-lang="sql" class="programlisting">EXISTS (SELECT 1 FROM ... WHERE <em class="replaceable"><code>subquery_where</code></em> AND <em class="replaceable"><code>outer_expr</code></em>=<em class="replaceable"><code>inner_expr</code></em>)
</pre><p>
          After the conversion, MySQL can use the pushed-down equality
          to limit the number of rows it must examine to evaluate the
          subquery.
        </p><p>
          More generally, a comparison of <em class="replaceable"><code>N</code></em>
          values to a subquery that returns
          <em class="replaceable"><code>N</code></em>-value rows is subject to the same
          conversion. If <em class="replaceable"><code>oe_i</code></em> and
          <em class="replaceable"><code>ie_i</code></em> represent corresponding outer
          and inner expression values, this subquery comparison:
        </p><pre data-lang="sql" class="programlisting">(<em class="replaceable"><code>oe_1</code></em>, ..., <em class="replaceable"><code>oe_N</code></em>) IN
  (SELECT <em class="replaceable"><code>ie_1</code></em>, ..., <em class="replaceable"><code>ie_N</code></em> FROM ... WHERE <em class="replaceable"><code>subquery_where</code></em>)
</pre><p>
          Becomes:
        </p><pre data-lang="sql" class="programlisting">EXISTS (SELECT 1 FROM ... WHERE <em class="replaceable"><code>subquery_where</code></em>
                          AND <em class="replaceable"><code>oe_1</code></em> = <em class="replaceable"><code>ie_1</code></em>
                          AND ...
                          AND <em class="replaceable"><code>oe_N</code></em> = <em class="replaceable"><code>ie_N</code></em>)
</pre><p>
          For simplicity, the following discussion assumes a single pair
          of outer and inner expression values.
        </p><p>
          The <span class="quote">“<span class="quote">pushdown</span>”</span> strategy just described works if
          either of these conditions is true:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <em class="replaceable"><code>outer_expr</code></em> and
              <em class="replaceable"><code>inner_expr</code></em> cannot be
              <code class="literal">NULL</code>.
            </p></li><li class="listitem"><p>
              You need not distinguish <code class="literal">NULL</code> from
              <code class="literal">FALSE</code> subquery results. If the subquery
              is a part of an <a class="link" href="functions.html#operator_or"><code class="literal">OR</code></a> or
              <a class="link" href="functions.html#operator_and"><code class="literal">AND</code></a> expression in the
              <code class="literal">WHERE</code> clause, MySQL assumes that you do
              not care. Another instance where the optimizer notices
              that <code class="literal">NULL</code> and <code class="literal">FALSE</code>
              subquery results need not be distinguished is this
              construct:
            </p><pre data-lang="sql" class="programlisting">... WHERE <em class="replaceable"><code>outer_expr</code></em> IN (<em class="replaceable"><code>subquery</code></em>)
</pre><p>
              In this case, the <code class="literal">WHERE</code> clause rejects
              the row whether <code class="literal">IN
              (<em class="replaceable"><code>subquery</code></em>)</code> returns
              <code class="literal">NULL</code> or <code class="literal">FALSE</code>.
</p></li></ul>
</div>
<p>
          Suppose that <em class="replaceable"><code>outer_expr</code></em> is known to
          be a non-<code class="literal">NULL</code> value but the subquery does
          not produce a row such that
          <em class="replaceable"><code>outer_expr</code></em> =
          <em class="replaceable"><code>inner_expr</code></em>. Then
          <code class="literal"><em class="replaceable"><code>outer_expr</code></em> IN (SELECT
          ...)</code> evaluates as follows:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <code class="literal">NULL</code>, if the
              <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> produces any row
              where <em class="replaceable"><code>inner_expr</code></em> is
              <code class="literal">NULL</code>
            </p></li><li class="listitem"><p>
              <code class="literal">FALSE</code>, if the
              <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> produces only
              non-<code class="literal">NULL</code> values or produces nothing
</p></li></ul>
</div>
<p>
          In this situation, the approach of looking for rows with
          <code class="literal"><em class="replaceable"><code>outer_expr</code></em> =
          <em class="replaceable"><code>inner_expr</code></em></code> is no longer
          valid. It is necessary to look for such rows, but if none are
          found, also look for rows where
          <em class="replaceable"><code>inner_expr</code></em> is
          <code class="literal">NULL</code>. Roughly speaking, the subquery can be
          converted to something like this:
        </p><pre data-lang="sql" class="programlisting">EXISTS (SELECT 1 FROM ... WHERE <em class="replaceable"><code>subquery_where</code></em> AND
        (<em class="replaceable"><code>outer_expr</code></em>=<em class="replaceable"><code>inner_expr</code></em> OR <em class="replaceable"><code>inner_expr</code></em> IS NULL))
</pre><p>
          The need to evaluate the extra <a class="link" href="functions.html#operator_is-null"><code class="literal">IS
          NULL</code></a> condition is why MySQL has the
          <a class="link" href="optimization.html#jointype_ref_or_null"><code class="literal">ref_or_null</code></a> access method:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN</code></strong>
       <strong class="userinput"><code>SELECT <em class="replaceable"><code>outer_expr</code></em> IN (SELECT t2.maybe_null_key</code></strong>
                             <strong class="userinput"><code>FROM t2, t3 WHERE ...)</code></strong>
       <strong class="userinput"><code>FROM t1;</code></strong>
*************************** 1. row ***************************
           id: 1
  select_type: PRIMARY
        table: t1
...
*************************** 2. row ***************************
           id: 2
  select_type: DEPENDENT SUBQUERY
        table: t2
         type: ref_or_null
possible_keys: maybe_null_key
          key: maybe_null_key
      key_len: 5
          ref: func
         rows: 2
        Extra: Using where; Using index
...
</pre><p>
          The <a class="link" href="optimization.html#jointype_unique_subquery"><code class="literal">unique_subquery</code></a> and
          <a class="link" href="optimization.html#jointype_index_subquery"><code class="literal">index_subquery</code></a>
          subquery-specific access methods also have <span class="quote">“<span class="quote">or
          <code class="literal">NULL</code></span>”</span> variants.
        </p><p>
          The additional <code class="literal">OR ... IS NULL</code> condition
          makes query execution slightly more complicated (and some
          optimizations within the subquery become inapplicable), but
          generally this is tolerable.
        </p><p>
          The situation is much worse when
          <em class="replaceable"><code>outer_expr</code></em> can be
          <code class="literal">NULL</code>. According to the SQL interpretation
          of <code class="literal">NULL</code> as <span class="quote">“<span class="quote">unknown value,</span>”</span>
          <code class="literal">NULL IN (SELECT <em class="replaceable"><code>inner_expr</code></em>
          ...)</code> should evaluate to:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <code class="literal">NULL</code>, if the
              <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> produces any rows
            </p></li><li class="listitem"><p>
              <code class="literal">FALSE</code>, if the
              <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> produces no rows
</p></li></ul>
</div>
<p>
          For proper evaluation, it is necessary to be able to check
          whether the <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> has produced
          any rows at all, so
          <code class="literal"><em class="replaceable"><code>outer_expr</code></em> =
          <em class="replaceable"><code>inner_expr</code></em></code> cannot be
          pushed down into the subquery. This is a problem because many
          real world subqueries become very slow unless the equality can
          be pushed down.
        </p><p>
          Essentially, there must be different ways to execute the
          subquery depending on the value of
          <em class="replaceable"><code>outer_expr</code></em>.
        </p><p>
          The optimizer chooses SQL compliance over speed, so it
          accounts for the possibility that
          <em class="replaceable"><code>outer_expr</code></em> might be
          <code class="literal">NULL</code>:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              If <em class="replaceable"><code>outer_expr</code></em> is
              <code class="literal">NULL</code>, to evaluate the following
              expression, it is necessary to execute the
              <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> to determine whether
              it produces any rows:
            </p><pre data-lang="sql" class="programlisting">NULL IN (SELECT <em class="replaceable"><code>inner_expr</code></em> FROM ... WHERE <em class="replaceable"><code>subquery_where</code></em>)
</pre><p>
              It is necessary to execute the original
              <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> here, without any
              pushed-down equalities of the kind mentioned previously.
            </p></li><li class="listitem"><p>
              On the other hand, when
              <em class="replaceable"><code>outer_expr</code></em> is not
              <code class="literal">NULL</code>, it is absolutely essential that
              this comparison:
            </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>outer_expr</code></em> IN (SELECT <em class="replaceable"><code>inner_expr</code></em> FROM ... WHERE <em class="replaceable"><code>subquery_where</code></em>)
</pre><p>
              Be converted to this expression that uses a pushed-down
              condition:
            </p><pre data-lang="sql" class="programlisting">EXISTS (SELECT 1 FROM ... WHERE <em class="replaceable"><code>subquery_where</code></em> AND <em class="replaceable"><code>outer_expr</code></em>=<em class="replaceable"><code>inner_expr</code></em>)
</pre><p>
              Without this conversion, subqueries will be slow.
</p></li></ul>
</div>
<p>
          To solve the dilemma of whether or not to push down conditions
          into the subquery, the conditions are wrapped within
          <span class="quote">“<span class="quote">trigger</span>”</span> functions. Thus, an expression of the
          following form:
        </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>outer_expr</code></em> IN (SELECT <em class="replaceable"><code>inner_expr</code></em> FROM ... WHERE <em class="replaceable"><code>subquery_where</code></em>)
</pre><p>
          Is converted into:
        </p><pre data-lang="sql" class="programlisting">EXISTS (SELECT 1 FROM ... WHERE <em class="replaceable"><code>subquery_where</code></em>
                          AND trigcond(<em class="replaceable"><code>outer_expr</code></em>=<em class="replaceable"><code>inner_expr</code></em>))
</pre><p>
          More generally, if the subquery comparison is based on several
          pairs of outer and inner expressions, the conversion takes
          this comparison:
        </p><pre data-lang="sql" class="programlisting">(<em class="replaceable"><code>oe_1</code></em>, ..., <em class="replaceable"><code>oe_N</code></em>) IN (SELECT <em class="replaceable"><code>ie_1</code></em>, ..., <em class="replaceable"><code>ie_N</code></em> FROM ... WHERE <em class="replaceable"><code>subquery_where</code></em>)
</pre><p>
          And converts it to this expression:
        </p><pre data-lang="sql" class="programlisting">EXISTS (SELECT 1 FROM ... WHERE <em class="replaceable"><code>subquery_where</code></em>
                          AND trigcond(<em class="replaceable"><code>oe_1</code></em>=<em class="replaceable"><code>ie_1</code></em>)
                          AND ...
                          AND trigcond(<em class="replaceable"><code>oe_N</code></em>=<em class="replaceable"><code>ie_N</code></em>)
       )
</pre><p>
          Each <code class="literal">trigcond(<em class="replaceable"><code>X</code></em>)</code>
          is a special function that evaluates to the following values:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <em class="replaceable"><code>X</code></em> when the
              <span class="quote">“<span class="quote">linked</span>”</span> outer expression
              <em class="replaceable"><code>oe_i</code></em> is not
              <code class="literal">NULL</code>
            </p></li><li class="listitem"><p>
              <code class="literal">TRUE</code> when the <span class="quote">“<span class="quote">linked</span>”</span>
              outer expression <em class="replaceable"><code>oe_i</code></em> is
              <code class="literal">NULL</code>
</p></li></ul>
</div>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">
<div class="admon-title">
Note
</div>
<p>
            Trigger functions are <span class="emphasis"><em>not</em></span> triggers of
            the kind that you create with <a class="link" href="sql-statements.html#create-trigger" title="13.1.22 CREATE TRIGGER Statement"><code class="literal">CREATE
            TRIGGER</code></a>.
</p>
</div>
<p>
          Equalities that are wrapped within
          <code class="literal">trigcond()</code> functions are not first class
          predicates for the query optimizer. Most optimizations cannot
          deal with predicates that may be turned on and off at query
          execution time, so they assume any
          <code class="literal">trigcond(<em class="replaceable"><code>X</code></em>)</code> to
          be an unknown function and ignore it. Triggered equalities can
          be used by those optimizations:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Reference optimizations:
              <code class="literal">trigcond(<em class="replaceable"><code>X</code></em>=<em class="replaceable"><code>Y</code></em>
              [OR <em class="replaceable"><code>Y</code></em> IS NULL])</code> can
              be used to construct
              <a class="link" href="optimization.html#jointype_ref"><code class="literal">ref</code></a>,
              <a class="link" href="optimization.html#jointype_eq_ref"><code class="literal">eq_ref</code></a>, or
              <a class="link" href="optimization.html#jointype_ref_or_null"><code class="literal">ref_or_null</code></a> table
              accesses.
            </p></li><li class="listitem"><p>
              Index lookup-based subquery execution engines:
              <code class="literal">trigcond(<em class="replaceable"><code>X</code></em>=<em class="replaceable"><code>Y</code></em>)</code>
              can be used to construct
              <a class="link" href="optimization.html#jointype_unique_subquery"><code class="literal">unique_subquery</code></a> or
              <a class="link" href="optimization.html#jointype_index_subquery"><code class="literal">index_subquery</code></a>
              accesses.
            </p></li><li class="listitem"><p>
              Table-condition generator: If the subquery is a join of
              several tables, the triggered condition is checked as soon
              as possible.
</p></li></ul>
</div>
<p>
          When the optimizer uses a triggered condition to create some
          kind of index lookup-based access (as for the first two items
          of the preceding list), it must have a fallback strategy for
          the case when the condition is turned off. This fallback
          strategy is always the same: Do a full table scan. In
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output, the fallback
          shows up as <code class="literal">Full scan on NULL key</code> in the
          <code class="literal">Extra</code> column:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN SELECT t1.col1,</code></strong>
       <strong class="userinput"><code>t1.col1 IN (SELECT t2.key1 FROM t2 WHERE t2.col2=t1.col2) FROM t1\G</code></strong>
*************************** 1. row ***************************
           id: 1
  select_type: PRIMARY
        table: t1
        ...
*************************** 2. row ***************************
           id: 2
  select_type: DEPENDENT SUBQUERY
        table: t2
         type: index_subquery
possible_keys: key1
          key: key1
      key_len: 5
          ref: func
         rows: 2
        Extra: Using where; Full scan on NULL key
</pre><p>
          If you run <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> followed by
          <a class="link" href="sql-statements.html#show-warnings" title="13.7.7.40 SHOW WARNINGS Statement"><code class="literal">SHOW WARNINGS</code></a>, you can see the
          triggered condition:
        </p><pre data-lang="none" class="programlisting">*************************** 1. row ***************************
  Level: Note
   Code: 1003
Message: select `test`.`t1`.`col1` AS `col1`,
         &lt;in_optimizer&gt;(`test`.`t1`.`col1`,
         &lt;exists&gt;(&lt;index_lookup&gt;(&lt;cache&gt;(`test`.`t1`.`col1`) in t2
         on key1 checking NULL
         where (`test`.`t2`.`col2` = `test`.`t1`.`col2`) having
         trigcond(&lt;is_not_null_test&gt;(`test`.`t2`.`key1`))))) AS
         `t1.col1 IN (select t2.key1 from t2 where t2.col2=t1.col2)`
         from `test`.`t1`</pre><p>
          The use of triggered conditions has some performance
          implications. A <code class="literal">NULL IN (SELECT ...)</code>
          expression now may cause a full table scan (which is slow)
          when it previously did not. This is the price paid for correct
          results (the goal of the trigger-condition strategy is to
          improve compliance, not speed).
        </p><p>
          For multiple-table subqueries, execution of <code class="literal">NULL IN
          (SELECT ...)</code> is particularly slow because the join
          optimizer does not optimize for the case where the outer
          expression is <code class="literal">NULL</code>. It assumes that
          subquery evaluations with <code class="literal">NULL</code> on the left
          side are very rare, even if there are statistics that indicate
          otherwise. On the other hand, if the outer expression might be
          <code class="literal">NULL</code> but never actually is, there is no
          performance penalty.
        </p><p>
          To help the query optimizer better execute your queries, use
          these suggestions:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Declare a column as <code class="literal">NOT NULL</code> if it
              really is. This also helps other aspects of the optimizer
              by simplifying condition testing for the column.
            </p></li><li class="listitem"><p>
              If you need not distinguish a <code class="literal">NULL</code> from
              <code class="literal">FALSE</code> subquery result, you can easily
              avoid the slow execution path. Replace a comparison that
              looks like this:
            </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>outer_expr</code></em> [NOT] IN (SELECT <em class="replaceable"><code>inner_expr</code></em> FROM ...)
</pre><p>
              with this expression:
            </p><pre data-lang="sql" class="programlisting">(<em class="replaceable"><code>outer_expr</code></em> IS NOT NULL) AND (<em class="replaceable"><code>outer_expr</code></em> [NOT] IN (SELECT <em class="replaceable"><code>inner_expr</code></em> FROM ...))
</pre><p>
              Then <code class="literal">NULL IN (SELECT ...)</code> is never
              evaluated because MySQL stops evaluating
              <a class="link" href="functions.html#operator_and"><code class="literal">AND</code></a> parts as soon as the
              expression result is clear.
            </p><p>
              Another possible rewrite:
            </p><pre data-lang="sql" class="programlisting">[NOT] EXISTS (SELECT <em class="replaceable"><code>inner_expr</code></em> FROM ...
        WHERE <em class="replaceable"><code>inner_expr</code></em>=<em class="replaceable"><code>outer_expr</code></em>)
</pre></li></ul>
</div>
<p>
          The <code class="literal">subquery_materialization_cost_based</code>
          flag of the <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a>
          system variable enables control over the choice between
          subquery materialization and
          <code class="literal">IN</code>-to-<code class="literal">EXISTS</code> subquery
          transformation. See
          <a class="xref" href="optimization.html#switchable-optimizations" title="8.9.2 Switchable Optimizations">Section 8.9.2, “Switchable Optimizations”</a>.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="derived-table-optimization"></a>8.2.2.4 Optimizing Derived Tables, View References, and Common Table Expressions
with Merging or Materialization</h4>
</div>
</div>
</div>
<a class="indexterm" name="idm46444351310400"></a><a class="indexterm" name="idm46444351308912"></a><a class="indexterm" name="idm46444351307424"></a><a class="indexterm" name="idm46444351305968"></a><a class="indexterm" name="idm46444351304480"></a><a class="indexterm" name="idm46444351302992"></a><a class="indexterm" name="idm46444351301488"></a><a class="indexterm" name="idm46444351300000"></a><a class="indexterm" name="idm46444351298512"></a><p>
          The optimizer can handle derived table references using two
          strategies (which also apply to view references and common
          table expressions):
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Merge the derived table into the outer query block
            </p></li><li class="listitem"><p>
              Materialize the derived table to an internal temporary
              table
</p></li></ul>
</div>
<p>
          Example 1:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM (SELECT * FROM t1) AS derived_t1;</pre><p>
          With merging of the derived table
          <code class="literal">derived_t1</code>, that query is executed similar
          to:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1;</pre><p>
          Example 2:
        </p><pre data-lang="sql" class="programlisting">SELECT *
  FROM t1 JOIN (SELECT t2.f1 FROM t2) AS derived_t2 ON t1.f2=derived_t2.f1
  WHERE t1.f1 &gt; 0;</pre><p>
          With merging of the derived table
          <code class="literal">derived_t2</code>, that query is executed similar
          to:
        </p><pre data-lang="sql" class="programlisting">SELECT t1.*, t2.f1
  FROM t1 JOIN t2 ON t1.f2=t2.f1
  WHERE t1.f1 &gt; 0;</pre><p>
          With materialization, <code class="literal">derived_t1</code> and
          <code class="literal">derived_t2</code> are each treated as a separate
          table within their respective queries.
        </p><p>
          The optimizer handles derived tables, view references, and
          common table expressions the same way: It avoids unnecessary
          materialization whenever possible, which enables pushing down
          conditions from the outer query to derived tables and produces
          more efficient execution plans. (For an example, see
          <a class="xref" href="optimization.html#subquery-materialization" title="8.2.2.2 Optimizing Subqueries with Materialization">Section 8.2.2.2, “Optimizing Subqueries with Materialization”</a>.)
        </p><p>
          If merging would result in an outer query block that
          references more than 61 base tables, the optimizer chooses
          materialization instead.
        </p><p>
          The optimizer propagates an <code class="literal">ORDER BY</code> clause
          in a derived table or view reference to the outer query block
          if these conditions are all true:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              The outer query is not grouped or aggregated.
            </p></li><li class="listitem"><p>
              The outer query does not specify
              <code class="literal">DISTINCT</code>, <code class="literal">HAVING</code>, or
              <code class="literal">ORDER BY</code>.
            </p></li><li class="listitem"><p>
              The outer query has this derived table or view reference
              as the only source in the <code class="literal">FROM</code> clause.
</p></li></ul>
</div>
<p>
          Otherwise, the optimizer ignores the <code class="literal">ORDER
          BY</code> clause.
        </p><p>
          The following means are available to influence whether the
          optimizer attempts to merge derived tables, view references,
          and common table expressions into the outer query block:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              The <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">MERGE</code></a> and
              <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">NO_MERGE</code></a> optimizer hints
              can be used. They apply assuming that no other rule
              prevents merging. See <a class="xref" href="optimization.html#optimizer-hints" title="8.9.3 Optimizer Hints">Section 8.9.3, “Optimizer Hints”</a>.
            </p></li><li class="listitem"><p>
              Similarly, you can use the
              <code class="literal">derived_merge</code> flag of the
              <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
              variable. See <a class="xref" href="optimization.html#switchable-optimizations" title="8.9.2 Switchable Optimizations">Section 8.9.2, “Switchable Optimizations”</a>.
              By default, the flag is enabled to permit merging.
              Disabling the flag prevents merging and avoids
              <a class="link" href="error-handling.html#error_er_update_table_used"><code class="literal">ER_UPDATE_TABLE_USED</code></a>
              errors.
            </p><p>
              The <code class="literal">derived_merge</code> flag also applies to
              views that contain no <code class="literal">ALGORITHM</code> clause.
              Thus, if an
              <a class="link" href="error-handling.html#error_er_update_table_used"><code class="literal">ER_UPDATE_TABLE_USED</code></a> error
              occurs for a view reference that uses an expression
              equivalent to the subquery, adding
              <code class="literal">ALGORITHM=TEMPTABLE</code> to the view
              definition prevents merging and takes precedence over the
              <code class="literal">derived_merge</code> value.
            </p></li><li class="listitem"><p>
              It is possible to disable merging by using in the subquery
              any constructs that prevent merging, although these are
              not as explicit in their effect on materialization.
              Constructs that prevent merging are the same for derived
              tables, common table expressions, and view references:
</p>
<div class="itemizedlist">
<a class="indexterm" name="idm46444351258544"></a><a class="indexterm" name="idm46444351257088"></a><ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                  Aggregate functions or window functions
                  (<a class="link" href="functions.html#function_sum"><code class="literal">SUM()</code></a>,
                  <a class="link" href="functions.html#function_min"><code class="literal">MIN()</code></a>,
                  <a class="link" href="functions.html#function_max"><code class="literal">MAX()</code></a>,
                  <a class="link" href="functions.html#function_count"><code class="literal">COUNT()</code></a>, and so forth)
                </p></li><li class="listitem"><p>
                  <code class="literal">DISTINCT</code>
                </p></li><li class="listitem"><p>
                  <code class="literal">GROUP BY</code>
                </p></li><li class="listitem"><p>
                  <code class="literal">HAVING</code>
                </p></li><li class="listitem"><p>
                  <code class="literal">LIMIT</code>
                </p></li><li class="listitem"><p>
                  <a class="link" href="sql-statements.html#union" title="13.2.10.3 UNION Clause"><code class="literal">UNION</code></a> or
                  <a class="link" href="sql-statements.html#union" title="13.2.10.3 UNION Clause"><code class="literal">UNION
                  ALL</code></a>
                </p></li><li class="listitem"><p>
                  Subqueries in the select list
                </p></li><li class="listitem"><p>
                  Assignments to user variables
                </p></li><li class="listitem"><p>
                  Refererences only to literal values (in this case,
                  there is no underlying table)
</p></li></ul>
</div>
</li></ul>
</div>
<p>
          If the optimizer chooses the materialization strategy rather
          than merging for a derived table, it handles the query as
          follows:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              The optimizer postpones derived table materialization
              until its contents are needed during query execution. This
              improves performance because delaying materialization may
              result in not having to do it at all. Consider a query
              that joins the result of a derived table to another table:
              If the optimizer processes that other table first and
              finds that it returns no rows, the join need not be
              carried out further and the optimizer can completely skip
              materializing the derived table.
            </p></li><li class="listitem"><p>
              During query execution, the optimizer may add an index to
              a derived table to speed up row retrieval from it.
</p></li></ul>
</div>
<p>
          Consider the following <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>
          statement, for a <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> query
          that contains a derived table:
        </p><pre data-lang="sql" class="programlisting">EXPLAIN SELECT * FROM (SELECT * FROM t1) AS derived_t1;</pre><p>
          The optimizer avoids materializing the derived table by
          delaying it until the result is needed during
          <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> execution. In this case,
          the query is not executed (because it occurs in an
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> statement), so the
          result is never needed.
        </p><p>
          Even for queries that are executed, delay of derived table
          materialization may enable the optimizer to avoid
          materialization entirely. When this happens, query execution
          is quicker by the time needed to perform materialization.
          Consider the following query, which joins the result of a
          derived table to another table:
        </p><pre data-lang="sql" class="programlisting">SELECT *
  FROM t1 JOIN (SELECT t2.f1 FROM t2) AS derived_t2
          ON t1.f2=derived_t2.f1
  WHERE t1.f1 &gt; 0;</pre><p>
          If the optimization processes <code class="literal">t1</code> first and
          the <code class="literal">WHERE</code> clause produces an empty result,
          the join must necessarily be empty and the derived table need
          not be materialized.
        </p><p>
          For cases when a derived table requires materialization, the
          optimizer may add an index to the materialized table to speed
          up access to it. If such an index enables
          <a class="link" href="optimization.html#jointype_ref"><code class="literal">ref</code></a> access to the table, it
          can greatly reduce amount of data read during query execution.
          Consider the following query:
        </p><pre data-lang="sql" class="programlisting">SELECT *
 FROM t1 JOIN (SELECT DISTINCT f1 FROM t2) AS derived_t2
         ON t1.f1=derived_t2.f1;</pre><p>
          The optimizer constructs an index over column
          <code class="literal">f1</code> from <code class="literal">derived_t2</code> if
          doing so would enable use of
          <a class="link" href="optimization.html#jointype_ref"><code class="literal">ref</code></a> access for the lowest
          cost execution plan. After adding the index, the optimizer can
          treat the materialized derived table the same as a regular
          table with an index, and it benefits similarly from the
          generated index. The overhead of index creation is negligible
          compared to the cost of query execution without the index. If
          <a class="link" href="optimization.html#jointype_ref"><code class="literal">ref</code></a> access would result in
          higher cost than some other access method, the optimizer
          creates no index and loses nothing.
        </p><p>
          For optimizer trace output, a merged derived table or view
          reference is not shown as a node. Only its underlying tables
          appear in the top query's plan.
        </p><p>
          What is true for materialization of derived tables is also
          true for common table expressions (CTEs). In addition, the
          following considerations pertain specifically to CTEs.
        </p><p>
          If a CTE is materialized by a query, it is materialized once
          for the query, even if the query references it several times.
        </p><p>
          A recursive CTE is always materialized.
        </p><p>
          If a CTE is materialized, the optimizer automatically adds
          relevant indexes if it estimates that indexing will speed up
          access by the top-level statement to the CTE. This is similar
          to automatic indexing of derived tables, except that if the
          CTE is referenced multiple times, the optimizer may create
          multiple indexes, to speed up access by each reference in the
          most appropriate way.
        </p><p>
          The <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">MERGE</code></a> and
          <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">NO_MERGE</code></a> optimizer hints can
          be applied to CTEs. Each CTE reference in the top-level
          statement can have its own hint, permitting CTE references to
          be selectively merged or materialized. The following statement
          uses hints to indicate that <code class="literal">cte1</code> should be
          merged and <code class="literal">cte2</code> should be materialized:
        </p><pre data-lang="sql" class="programlisting">WITH
  cte1 AS (SELECT a, b FROM table1),
  cte2 AS (SELECT c, d FROM table2)
SELECT /*+ MERGE(cte1) NO_MERGE(cte2) */ cte1.b, cte2.d
FROM cte1 JOIN cte2
WHERE cte1.a = cte2.c;</pre><p>
          The <code class="literal">ALGORITHM</code> clause for
          <a class="link" href="sql-statements.html#create-view" title="13.1.23 CREATE VIEW Statement"><code class="literal">CREATE VIEW</code></a> does not affect
          materialization for any <a class="link" href="sql-statements.html#with" title="13.2.15 WITH (Common Table Expressions)"><code class="literal">WITH</code></a>
          clause preceding the <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>
          statement in the view definition. Consider this statement:
        </p><pre data-lang="sql" class="programlisting">CREATE ALGORITHM={TEMPTABLE|MERGE} VIEW v1 AS WITH ... SELECT ...</pre><p>
          The <code class="literal">ALGORITHM</code> value affects materialization
          only of the <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>, not the
          <a class="link" href="sql-statements.html#with" title="13.2.15 WITH (Common Table Expressions)"><code class="literal">WITH</code></a> clause.
        </p><p>
          Prior to MySQL 8.0.16, if
          <a class="link" href="server-administration.html#sysvar_internal_tmp_disk_storage_engine"><code class="literal">internal_tmp_disk_storage_engine=MYISAM</code></a>,
          an error occurred for any attempt to materialize a CTE using
          an on-disk temporary table, since for CTEs, the storage engine
          used for on-disk internal temporary tables could not be
          <code class="literal">MyISAM</code>. Beginning with MySQL 8.0.16, this
          is no longer an issue, since <code class="literal">TempTable</code> now
          always uses <code class="literal">InnoDB</code> for on-disk internal
          temporary tables.
        </p><p>
          As mentioned previously, a CTE, if materialized, is
          materialized once, even if referenced multiple times. To
          indicate one-time materialization, optimizer trace output
          contains an occurrence of
          <code class="literal">creating_tmp_table</code> plus one or more
          occurrences of <code class="literal">reusing_tmp_table</code>.
        </p><p>
          CTEs are similar to derived tables, for which the
          <code class="literal">materialized_from_subquery</code> node follows the
          reference. This is true for a CTE that is referenced multiple
          times, so there is no duplication of
          <code class="literal">materialized_from_subquery</code> nodes (which
          would give the impression that the subquery is executed
          multiple times, and produce unnecessarily verbose output).
          Only one reference to the CTE has a complete
          <code class="literal">materialized_from_subquery</code> node with the
          description of its subquery plan. Other references have a
          reduced <code class="literal">materialized_from_subquery</code> node.
          The same idea applies to
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output in
          <code class="literal">TRADITIONAL</code> format: Subqueries for other
          references are not shown.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="information-schema-optimization"></a>8.2.3 Optimizing INFORMATION_SCHEMA Queries</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444351183760"></a><a class="indexterm" name="idm46444351182256"></a><p>
        Applications that monitor databases may make frequent use of
        <code class="literal">INFORMATION_SCHEMA</code> tables. To write queries
        for these tables most efficiently, use the following general
        guidelines:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Try to query only <code class="literal">INFORMATION_SCHEMA</code>
            tables that are views on data dictionary tables.
          </p></li><li class="listitem"><p>
            Try to query only for static metadata. Selecting columns or
            using retrieval conditions for dynamic metadata along with
            static metadata adds overhead to process the dynamic
            metadata.
</p></li></ul>
</div>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">
<div class="admon-title">
Note
</div>
<p>
          Comparison behavior for database and table names in
          <code class="literal">INFORMATION_SCHEMA</code> queries might differ
          from what you expect. For details, see
          <a class="xref" href="charset.html#charset-collation-information-schema" title="10.8.7 Using Collation in INFORMATION_SCHEMA Searches">Section 10.8.7, “Using Collation in INFORMATION_SCHEMA Searches”</a>.
</p>
</div>
<p>
        These <code class="literal">INFORMATION_SCHEMA</code> tables are
        implemented as views on data dictionary tables, so queries on
        them retrieve information from the data dictionary:
      </p><pre data-lang="none" class="programlisting">CHARACTER_SETS
CHECK_CONSTRAINTS
COLLATIONS
COLLATION_CHARACTER_SET_APPLICABILITY
COLUMNS
EVENTS
FILES
INNODB_COLUMNS
INNODB_DATAFILES
INNODB_FIELDS
INNODB_FOREIGN
INNODB_FOREIGN_COLS
INNODB_INDEXES
INNODB_TABLES
INNODB_TABLESPACES
INNODB_TABLESPACES_BRIEF
INNODB_TABLESTATS
KEY_COLUMN_USAGE
PARAMETERS
PARTITIONS
REFERENTIAL_CONSTRAINTS
RESOURCE_GROUPS
ROUTINES
SCHEMATA
STATISTICS
TABLES
TABLE_CONSTRAINTS
TRIGGERS
VIEWS
VIEW_ROUTINE_USAGE
VIEW_TABLE_USAGE</pre><p>
        Some types of values, even for a non-view
        <code class="literal">INFORMATION_SCHEMA</code> table, are retrieved by
        lookups from the data dictionary. This includes values such as
        database and table names, table types, and storage engines.
      </p><p>
        Some <code class="literal">INFORMATION_SCHEMA</code> tables contain
        columns that provide table statistics:
      </p><pre data-lang="none" class="programlisting">STATISTICS.CARDINALITY
TABLES.AUTO_INCREMENT
TABLES.AVG_ROW_LENGTH
TABLES.CHECKSUM
TABLES.CHECK_TIME
TABLES.CREATE_TIME
TABLES.DATA_FREE
TABLES.DATA_LENGTH
TABLES.INDEX_LENGTH
TABLES.MAX_DATA_LENGTH
TABLES.TABLE_ROWS
TABLES.UPDATE_TIME</pre><p>
        Those columns represent dynamic table metadata; that is,
        information that changes as table contents change.
      </p><p>
        By default, MySQL retrieves cached values for those columns from
        the <code class="literal">mysql.index_stats</code> and
        <code class="literal">mysql.table_stats</code> dictionary tables when the
        columns are queried, which is more efficient than retrieving
        statistics directly from the storage engine. If cached
        statistics are not available or have expired, MySQL retrieves
        the latest statistics from the storage engine and caches them in
        the <code class="literal">mysql.index_stats</code> and
        <code class="literal">mysql.table_stats</code> dictionary tables.
        Subsequent queries retrieve the cached statistics until the
        cached statistics expire.
      </p><p>
        The
        <a class="link" href="server-administration.html#sysvar_information_schema_stats_expiry"><code class="literal">information_schema_stats_expiry</code></a>
        session variable defines the period of time before cached
        statistics expire. The default is 86400 seconds (24 hours), but
        the time period can be extended to as much as one year.
      </p><p>
        To update cached values at any time for a given table, use
        <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE TABLE</code></a>.
      </p><p>
        Querying statistics columns does not store or update statistics
        in the <code class="literal">mysql.index_stats</code> and
        <code class="literal">mysql.table_stats</code> dictionary tables under
        these circumstances:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            When cached statistics have not expired.
          </p></li><li class="listitem"><p>
            When
            <a class="link" href="server-administration.html#sysvar_information_schema_stats_expiry"><code class="literal">information_schema_stats_expiry</code></a>
            is set to 0.
          </p></li><li class="listitem"><p>
            When the server is started in
            <a class="link" href="server-administration.html#sysvar_read_only"><code class="literal">read_only</code></a>,
            <a class="link" href="server-administration.html#sysvar_super_read_only"><code class="literal">super_read_only</code></a>,
            <a class="link" href="server-administration.html#sysvar_transaction_read_only"><code class="literal">transaction_read_only</code></a>, or
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_read_only"><code class="literal">innodb_read_only</code></a> mode.
          </p></li><li class="listitem"><p>
            When the query also fetches Performance Schema data.
</p></li></ul>
</div>
<p>
        <a class="link" href="server-administration.html#sysvar_information_schema_stats_expiry"><code class="literal">information_schema_stats_expiry</code></a>
        is a session variable, and each client session can define its
        own expiration value. Statistics that are retrieved from the
        storage engine and cached by one session are available to other
        sessions.
</p>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
          If the <a class="link" href="innodb-storage-engine.html#sysvar_innodb_read_only"><code class="literal">innodb_read_only</code></a>
          system variable is enabled, <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE
          TABLE</code></a> may fail because it cannot update statistics
          tables in the data dictionary, which use
          <code class="literal">InnoDB</code>. For <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE
          TABLE</code></a> operations that update the key distribution,
          failure may occur even if the operation updates the table
          itself (for example, if it is a <code class="literal">MyISAM</code>
          table). To obtain the updated distribution statistics, set
          <a class="link" href="server-administration.html#sysvar_information_schema_stats_expiry"><code class="literal">information_schema_stats_expiry=0</code></a>.
</p>
</div>
<p>
        For <code class="literal">INFORMATION_SCHEMA</code> tables implemented as
        views on data dictionary tables, indexes on the underlying data
        dictionary tables permit the optimizer to construct efficient
        query execution plans. To see the choices made by the optimizer,
        use <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>. To also see the
        query used by the server to execute an
        <code class="literal">INFORMATION_SCHEMA</code> query, use
        <a class="link" href="sql-statements.html#show-warnings" title="13.7.7.40 SHOW WARNINGS Statement"><code class="literal">SHOW WARNINGS</code></a> immediately
        following <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>.
      </p><p>
        Consider this statement, which identifies collations for the
        <code class="literal">utf8mb4</code> character set:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT COLLATION_NAME</code></strong>
       <strong class="userinput"><code>FROM INFORMATION_SCHEMA.COLLATION_CHARACTER_SET_APPLICABILITY</code></strong>
       <strong class="userinput"><code>WHERE CHARACTER_SET_NAME = 'utf8mb4';</code></strong>
+----------------------------+
| COLLATION_NAME             |
+----------------------------+
| utf8mb4_general_ci         |
| utf8mb4_bin                |
| utf8mb4_unicode_ci         |
| utf8mb4_icelandic_ci       |
| utf8mb4_latvian_ci         |
| utf8mb4_romanian_ci        |
| utf8mb4_slovenian_ci       |
...
</pre><p>
        How does the server process that statement? To find out, use
        <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN SELECT COLLATION_NAME</code></strong>
       <strong class="userinput"><code>FROM INFORMATION_SCHEMA.COLLATION_CHARACTER_SET_APPLICABILITY</code></strong>
       <strong class="userinput"><code>WHERE CHARACTER_SET_NAME = 'utf8mb4'\G</code></strong>
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: cs
   partitions: NULL
         type: const
possible_keys: PRIMARY,name
          key: name
      key_len: 194
          ref: const
         rows: 1
     filtered: 100.00
        Extra: Using index
*************************** 2. row ***************************
           id: 1
  select_type: SIMPLE
        table: col
   partitions: NULL
         type: ref
possible_keys: character_set_id
          key: character_set_id
      key_len: 8
          ref: const
         rows: 68
     filtered: 100.00
        Extra: NULL
2 rows in set, 1 warning (0.01 sec)
</pre><p>
        To see the query used to statisfy that statement, use
        <a class="link" href="sql-statements.html#show-warnings" title="13.7.7.40 SHOW WARNINGS Statement"><code class="literal">SHOW WARNINGS</code></a>:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SHOW WARNINGS\G</code></strong>
*************************** 1. row ***************************
  Level: Note
   Code: 1003
Message: /* select#1 */ select `mysql`.`col`.`name` AS `COLLATION_NAME`
         from `mysql`.`character_sets` `cs`
         join `mysql`.`collations` `col`
         where ((`mysql`.`col`.`character_set_id` = '45')
         and ('utf8mb4' = 'utf8mb4'))
</pre><p>
        As indicated by <a class="link" href="sql-statements.html#show-warnings" title="13.7.7.40 SHOW WARNINGS Statement"><code class="literal">SHOW WARNINGS</code></a>,
        the server handles the query on
        <a class="link" href="information-schema.html#collation-character-set-applicability-table" title="25.7 The INFORMATION_SCHEMA COLLATION_CHARACTER_SET_APPLICABILITY Table"><code class="literal">COLLATION_CHARACTER_SET_APPLICABILITY</code></a>
        as a query on the <code class="literal">character_sets</code> and
        <code class="literal">collations</code> data dictionary tables in the
        <code class="literal">mysql</code> system database.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="performance-schema-optimization"></a>8.2.4 Optimizing Performance Schema Queries</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444351111872"></a><a class="indexterm" name="idm46444351110368"></a><p>
        Applications that monitor databases may make frequent use of
        Performance Schema tables. To write queries for these tables
        most efficiently, take advantage of their indexes. For example,
        include a <code class="literal">WHERE</code> clause that restricts
        retrieved rows based on comparison to specific values in an
        indexed column.
      </p><p>
        Most Performance Schema tables have indexes. Tables that do not
        are those that normally contain few rows or are unlikely to be
        queried frequently. Performance Schema indexes give the
        optimizer access to execution plans other than full table scans.
        These indexes also improve performance for related objects, such
        as <a class="link" href="sys-schema.html" title="Chapter 27 MySQL sys Schema"><code class="literal">sys</code></a> schema views that use those
        tables.
      </p><p>
        To see whether a given Performance Schema table has indexes and
        what they are, use <a class="link" href="sql-statements.html#show-index" title="13.7.7.22 SHOW INDEX Statement"><code class="literal">SHOW INDEX</code></a> or
        <a class="link" href="sql-statements.html#show-create-table" title="13.7.7.10 SHOW CREATE TABLE Statement"><code class="literal">SHOW CREATE TABLE</code></a>:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SHOW INDEX FROM performance_schema.accounts\G</code></strong>
*************************** 1. row ***************************
        Table: accounts
   Non_unique: 0
     Key_name: ACCOUNT
 Seq_in_index: 1
  Column_name: USER
    Collation: NULL
  Cardinality: NULL
     Sub_part: NULL
       Packed: NULL
         Null: YES
   Index_type: HASH
      Comment:
Index_comment:
      Visible: YES
*************************** 2. row ***************************
        Table: accounts
   Non_unique: 0
     Key_name: ACCOUNT
 Seq_in_index: 2
  Column_name: HOST
    Collation: NULL
  Cardinality: NULL
     Sub_part: NULL
       Packed: NULL
         Null: YES
   Index_type: HASH
      Comment:
Index_comment:
      Visible: YES

mysql&gt; <strong class="userinput"><code>SHOW CREATE TABLE performance_schema.rwlock_instances\G</code></strong>
*************************** 1. row ***************************
       Table: rwlock_instances
Create Table: CREATE TABLE `rwlock_instances` (
  `NAME` varchar(128) NOT NULL,
  `OBJECT_INSTANCE_BEGIN` bigint(20) unsigned NOT NULL,
  `WRITE_LOCKED_BY_THREAD_ID` bigint(20) unsigned DEFAULT NULL,
  `READ_LOCKED_BY_COUNT` int(10) unsigned NOT NULL,
  PRIMARY KEY (`OBJECT_INSTANCE_BEGIN`),
  KEY `NAME` (`NAME`),
  KEY `WRITE_LOCKED_BY_THREAD_ID` (`WRITE_LOCKED_BY_THREAD_ID`)
) ENGINE=PERFORMANCE_SCHEMA DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci
</pre><p>
        To see the execution plan for a Performance Schema query and
        whether it uses any indexes, use
        <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN SELECT * FROM performance_schema.accounts</code></strong>
       <strong class="userinput"><code>WHERE (USER,HOST) = ('root','localhost')\G</code></strong>
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: accounts
   partitions: NULL
         type: const
possible_keys: ACCOUNT
          key: ACCOUNT
      key_len: 278
          ref: const,const
         rows: 1
     filtered: 100.00
        Extra: NULL
</pre><p>
        The <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output indicates that
        the optimizer uses the <a class="link" href="performance-schema.html#accounts-table" title="26.12.8.1 The accounts Table"><code class="literal">accounts</code></a>
        table <code class="literal">ACCOUNT</code> index that comprises the
        <code class="literal">USER</code> and <code class="literal">HOST</code> columns.
      </p><p>
        Performance Schema indexes are virtual: They are a construct of
        the Performance Schema storage engine and use no memory or disk
        storage. The Performance Schema reports index information to the
        optimizer so that it can construct efficient execution plans.
        The Performance Schema in turn uses optimizer information about
        what to look for (for example, a particular key value), so that
        it can perform efficient lookups without building actual index
        structures. This implementation provides two important benefits:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            It entirely avoids the maintenance cost normally incurred
            for tables that undergo frequent updates.
          </p></li><li class="listitem"><p>
            It reduces at an early stage of query execution the amount
            of data retrieved. For conditions on the indexed columns,
            the Performance Schema efficiently returns only table rows
            that satisfy the query conditions. Without an index, the
            Performance Schema would return all rows in the table,
            requiring that the optimizer later evaluate the conditions
            against each row to produce the final result.
</p></li></ul>
</div>
<p>
        Performance Schema indexes are predefined and cannot be dropped,
        added, or altered.
      </p><p>
        Performance Schema indexes are similar to hash indexes. For
        example:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            They are used only for equality comparisons that use the
            <code class="literal">=</code> or <code class="literal">&lt;=&gt;</code>
            operators.
          </p></li><li class="listitem"><p>
            They are unordered. If a query result must have specific row
            ordering characteristics, include an <code class="literal">ORDER
            BY</code> clause.
</p></li></ul>
</div>
<p>
        For additional information about hash indexes, see
        <a class="xref" href="optimization.html#index-btree-hash" title="8.3.9 Comparison of B-Tree and Hash Indexes">Section 8.3.9, “Comparison of B-Tree and Hash Indexes”</a>.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="data-change-optimization"></a>8.2.5 Optimizing Data Change Statements</h3>

</div>

</div>

</div>
<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#insert-optimization">8.2.5.1 Optimizing INSERT Statements</a></span></dt><dt><span class="section"><a href="optimization.html#update-optimization">8.2.5.2 Optimizing UPDATE Statements</a></span></dt><dt><span class="section"><a href="optimization.html#delete-optimization">8.2.5.3 Optimizing DELETE Statements</a></span></dt></dl>
</div>
<a class="indexterm" name="idm46444351076976"></a><p>
        This section explains how to speed up data change statements:
        <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>,
        <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a>, and
        <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a>. Traditional OLTP
        applications and modern web applications typically do many small
        data change operations, where concurrency is vital. Data
        analysis and reporting applications typically run data change
        operations that affect many rows at once, where the main
        considerations is the I/O to write large amounts of data and
        keep indexes up-to-date. For inserting and updating large
        volumes of data (known in the industry as ETL, for
        <span class="quote">“<span class="quote">extract-transform-load</span>”</span>), sometimes you use other
        SQL statements or external commands, that mimic the effects of
        <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>,
        <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a>, and
        <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a> statements.
</p>
<div class="section">

<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="insert-optimization"></a>8.2.5.1 Optimizing INSERT Statements</h4>
</div>
</div>
</div>
<a class="indexterm" name="idm46444351064944"></a><a class="indexterm" name="idm46444351063904"></a><a class="indexterm" name="idm46444351062416"></a><a class="indexterm" name="idm46444351060928"></a><p>
          To optimize insert speed, combine many small operations into a
          single large operation. Ideally, you make a single connection,
          send the data for many new rows at once, and delay all index
          updates and consistency checking until the very end.
        </p><p>
          The time required for inserting a row is determined by the
          following factors, where the numbers indicate approximate
          proportions:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Connecting: (3)
            </p></li><li class="listitem"><p>
              Sending query to server: (2)
            </p></li><li class="listitem"><p>
              Parsing query: (2)
            </p></li><li class="listitem"><p>
              Inserting row: (1 × size of row)
            </p></li><li class="listitem"><p>
              Inserting indexes: (1 × number of indexes)
            </p></li><li class="listitem"><p>
              Closing: (1)
</p></li></ul>
</div>
<p>
          This does not take into consideration the initial overhead to
          open tables, which is done once for each concurrently running
          query.
        </p><p>
          The size of the table slows down the insertion of indexes by
          log <em class="replaceable"><code>N</code></em>, assuming B-tree indexes.
        </p><p>
          You can use the following methods to speed up inserts:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              If you are inserting many rows from the same client at the
              same time, use <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>
              statements with multiple <code class="literal">VALUES</code> lists
              to insert several rows at a time. This is considerably
              faster (many times faster in some cases) than using
              separate single-row <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>
              statements. If you are adding data to a nonempty table,
              you can tune the
              <a class="link" href="server-administration.html#sysvar_bulk_insert_buffer_size"><code class="literal">bulk_insert_buffer_size</code></a>
              variable to make data insertion even faster. See
              <a class="xref" href="server-administration.html#server-system-variables" title="5.1.8 Server System Variables">Section 5.1.8, “Server System Variables”</a>.
            </p></li><li class="listitem"><p>
              When loading a table from a text file, use
              <a class="link" href="sql-statements.html#load-data" title="13.2.7 LOAD DATA Statement"><code class="literal">LOAD DATA</code></a>. This is usually
              20 times faster than using
              <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a> statements. See
              <a class="xref" href="sql-statements.html#load-data" title="13.2.7 LOAD DATA Statement">Section 13.2.7, “LOAD DATA Statement”</a>.
            </p></li><li class="listitem"><p>
              Take advantage of the fact that columns have default
              values. Insert values explicitly only when the value to be
              inserted differs from the default. This reduces the
              parsing that MySQL must do and improves the insert speed.
            </p></li><li class="listitem"><p>
              See <a class="xref" href="optimization.html#optimizing-innodb-bulk-data-loading" title="8.5.5 Bulk Data Loading for InnoDB Tables">Section 8.5.5, “Bulk Data Loading for InnoDB Tables”</a>
              for tips specific to <code class="literal">InnoDB</code> tables.
            </p></li><li class="listitem"><p>
              See <a class="xref" href="optimization.html#optimizing-myisam-bulk-data-loading" title="8.6.2 Bulk Data Loading for MyISAM Tables">Section 8.6.2, “Bulk Data Loading for MyISAM Tables”</a>
              for tips specific to <code class="literal">MyISAM</code> tables.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="update-optimization"></a>8.2.5.2 Optimizing UPDATE Statements</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444351032592"></a><p>
          An update statement is optimized like a
          <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> query with the
          additional overhead of a write. The speed of the write depends
          on the amount of data being updated and the number of indexes
          that are updated. Indexes that are not changed do not get
          updated.
        </p><p>
          Another way to get fast updates is to delay updates and then
          do many updates in a row later. Performing multiple updates
          together is much quicker than doing one at a time if you lock
          the table.
        </p><p>
          For a <code class="literal">MyISAM</code> table that uses dynamic row
          format, updating a row to a longer total length may split the
          row. If you do this often, it is very important to use
          <a class="link" href="sql-statements.html#optimize-table" title="13.7.3.4 OPTIMIZE TABLE Statement"><code class="literal">OPTIMIZE TABLE</code></a> occasionally.
          See <a class="xref" href="sql-statements.html#optimize-table" title="13.7.3.4 OPTIMIZE TABLE Statement">Section 13.7.3.4, “OPTIMIZE TABLE Statement”</a>.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="delete-optimization"></a>8.2.5.3 Optimizing DELETE Statements</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444351024016"></a><p>
          The time required to delete individual rows in a
          <code class="literal">MyISAM</code> table is exactly proportional to the
          number of indexes. To delete rows more quickly, you can
          increase the size of the key cache by increasing the
          <a class="link" href="server-administration.html#sysvar_key_buffer_size"><code class="literal">key_buffer_size</code></a> system
          variable. See <a class="xref" href="server-administration.html#server-configuration" title="5.1.1 Configuring the Server">Section 5.1.1, “Configuring the Server”</a>.
        </p><p>
          To delete all rows from a <code class="literal">MyISAM</code> table,
          <code class="literal">TRUNCATE TABLE
          <em class="replaceable"><code>tbl_name</code></em></code> is faster than
          <code class="literal">DELETE FROM
          <em class="replaceable"><code>tbl_name</code></em></code>. Truncate
          operations are not transaction-safe; an error occurs when
          attempting one in the course of an active transaction or
          active table lock. See <a class="xref" href="sql-statements.html#truncate-table" title="13.1.37 TRUNCATE TABLE Statement">Section 13.1.37, “TRUNCATE TABLE Statement”</a>.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="permission-optimization"></a>8.2.6 Optimizing Database Privileges</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444351013872"></a><a class="indexterm" name="idm46444351012416"></a><p>
        The more complex your privilege setup, the more overhead applies
        to all SQL statements. Simplifying the privileges established by
        <a class="link" href="sql-statements.html#grant" title="13.7.1.6 GRANT Statement"><code class="literal">GRANT</code></a> statements enables MySQL to
        reduce permission-checking overhead when clients execute
        statements. For example, if you do not grant any table-level or
        column-level privileges, the server need not ever check the
        contents of the <code class="literal">tables_priv</code> and
        <code class="literal">columns_priv</code> tables. Similarly, if you place
        no resource limits on any accounts, the server does not have to
        perform resource counting. If you have a very high
        statement-processing load, consider using a simplified grant
        structure to reduce permission-checking overhead.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="miscellaneous-optimization-tips"></a>8.2.7 Other Optimization Tips</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444351005808"></a><a class="indexterm" name="idm46444351004320"></a><p>
        This section lists a number of miscellaneous tips for improving
        query processing speed:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            If your application makes several database requests to
            perform related updates, combining the statements into a
            stored routine can help performance. Similarly, if your
            application computes a single result based on several column
            values or large volumes of data, combining the computation
            into a UDF (user-defined function) can help performance. The
            resulting fast database operations are then available to be
            reused by other queries, applications, and even code written
            in different programming languages. See
            <a class="xref" href="stored-objects.html#stored-routines" title="24.2 Using Stored Routines">Section 24.2, “Using Stored Routines”</a> and
            <a class="xref" href="extending-mysql.html#adding-functions" title="29.4 Adding Functions to MySQL">Section 29.4, “Adding Functions to MySQL”</a> for more information.
          </p></li><li class="listitem"><p>
            To fix any compression issues that occur with
            <code class="literal">ARCHIVE</code> tables, use
            <a class="link" href="sql-statements.html#optimize-table" title="13.7.3.4 OPTIMIZE TABLE Statement"><code class="literal">OPTIMIZE TABLE</code></a>. See
            <a class="xref" href="storage-engines.html#archive-storage-engine" title="16.5 The ARCHIVE Storage Engine">Section 16.5, “The ARCHIVE Storage Engine”</a>.
          </p></li><li class="listitem"><p>
            If possible, classify reports as <span class="quote">“<span class="quote">live</span>”</span> or as
            <span class="quote">“<span class="quote">statistical</span>”</span>, where data needed for
            statistical reports is created only from summary tables that
            are generated periodically from the live data.
          </p></li><li class="listitem"><p>
            If you have data that does not conform well to a
            rows-and-columns table structure, you can pack and store
            data into a <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a> column. In
            this case, you must provide code in your application to pack
            and unpack information, but this might save I/O operations
            to read and write the sets of related values.
          </p></li><li class="listitem"><p>
            With Web servers, store images and other binary assets as
            files, with the path name stored in the database rather than
            the file itself. Most Web servers are better at caching
            files than database contents, so using files is generally
            faster. (Although you must handle backups and storage issues
            yourself in this case.)
          </p></li><li class="listitem"><p>
            If you need really high speed, look at the low-level MySQL
            interfaces. For example, by accessing the MySQL
            <a class="link" href="innodb-storage-engine.html" title="Chapter 15 The InnoDB Storage Engine"><code class="literal">InnoDB</code></a> or
            <a class="link" href="storage-engines.html#myisam-storage-engine" title="16.2 The MyISAM Storage Engine"><code class="literal">MyISAM</code></a> storage engine directly,
            you could get a substantial speed increase compared to using
            the SQL interface.
          </p><p>
            Similarly, for databases using the
            <a class="link" href="mysql-cluster.html" title="Chapter 22 MySQL NDB Cluster 8.0"><code class="literal">NDBCLUSTER</code></a> storage engine, you
            may wish to investigate possible use of the NDB API (see
            <a class="ulink" href="https://dev.mysql.com/doc/ndbapi/en/" target="_top">MySQL NDB Cluster API Developer Guide</a>).
          </p></li><li class="listitem"><p>
            Replication can provide a performance benefit for some
            operations. You can distribute client retrievals among
            replication servers to split up the load. To avoid slowing
            down the master while making backups, you can make backups
            using a slave server. See <a class="xref" href="replication.html" title="Chapter 17 Replication">Chapter 17, <i>Replication</i></a>.
</p></li></ul>
</div>

</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a name="optimization-indexes"></a>8.3 Optimization and Indexes</h2>

</div>

</div>

</div>
<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#mysql-indexes">8.3.1 How MySQL Uses Indexes</a></span></dt><dt><span class="section"><a href="optimization.html#primary-key-optimization">8.3.2 Primary Key Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#spatial-index-optimization">8.3.3 SPATIAL Index Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#foreign-key-optimization">8.3.4 Foreign Key Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#column-indexes">8.3.5 Column Indexes</a></span></dt><dt><span class="section"><a href="optimization.html#multiple-column-indexes">8.3.6 Multiple-Column Indexes</a></span></dt><dt><span class="section"><a href="optimization.html#verifying-index-usage">8.3.7 Verifying Index Usage</a></span></dt><dt><span class="section"><a href="optimization.html#index-statistics">8.3.8 InnoDB and MyISAM Index Statistics Collection</a></span></dt><dt><span class="section"><a href="optimization.html#index-btree-hash">8.3.9 Comparison of B-Tree and Hash Indexes</a></span></dt><dt><span class="section"><a href="optimization.html#index-extensions">8.3.10 Use of Index Extensions</a></span></dt><dt><span class="section"><a href="optimization.html#generated-column-index-optimizations">8.3.11 Optimizer Use of Generated Column Indexes</a></span></dt><dt><span class="section"><a href="optimization.html#invisible-indexes">8.3.12 Invisible Indexes</a></span></dt><dt><span class="section"><a href="optimization.html#descending-indexes">8.3.13 Descending Indexes</a></span></dt><dt><span class="section"><a href="optimization.html#timestamp-lookups">8.3.14 Indexed Lookups from TIMESTAMP Columns</a></span></dt></dl>
</div>
<a class="indexterm" name="idm46444350980880"></a><p>
      The best way to improve the performance of
      <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> operations is to create
      indexes on one or more of the columns that are tested in the
      query. The index entries act like pointers to the table rows,
      allowing the query to quickly determine which rows match a
      condition in the <code class="literal">WHERE</code> clause, and retrieve the
      other column values for those rows. All MySQL data types can be
      indexed.
    </p><p>
      Although it can be tempting to create an indexes for every
      possible column used in a query, unnecessary indexes waste space
      and waste time for MySQL to determine which indexes to use.
      Indexes also add to the cost of inserts, updates, and deletes
      because each index must be updated. You must find the right
      balance to achieve fast queries using the optimal set of indexes.
</p>
<div class="section">

<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="mysql-indexes"></a>8.3.1 How MySQL Uses Indexes</h3>
</div>
</div>
</div>
<a class="indexterm" name="idm46444350974752"></a><p>
        Indexes are used to find rows with specific column values
        quickly. Without an index, MySQL must begin with the first row
        and then read through the entire table to find the relevant
        rows. The larger the table, the more this costs. If the table
        has an index for the columns in question, MySQL can quickly
        determine the position to seek to in the middle of the data file
        without having to look at all the data. This is much faster than
        reading every row sequentially.
      </p><p>
        Most MySQL indexes (<code class="literal">PRIMARY KEY</code>,
        <code class="literal">UNIQUE</code>, <code class="literal">INDEX</code>, and
        <code class="literal">FULLTEXT</code>) are stored in
        <a class="link" href="glossary.html#glos_b_tree" title="B-tree">B-trees</a>. Exceptions: Indexes
        on spatial data types use R-trees; <code class="literal">MEMORY</code>
        tables also support <a class="link" href="glossary.html#glos_hash_index" title="hash index">hash
        indexes</a>; <code class="literal">InnoDB</code> uses inverted lists
        for <code class="literal">FULLTEXT</code> indexes.
      </p><p>
        In general, indexes are used as described in the following
        discussion. Characteristics specific to hash indexes (as used in
        <code class="literal">MEMORY</code> tables) are described in
        <a class="xref" href="optimization.html#index-btree-hash" title="8.3.9 Comparison of B-Tree and Hash Indexes">Section 8.3.9, “Comparison of B-Tree and Hash Indexes”</a>.
      </p><p>
        MySQL uses indexes for these operations:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            To find the rows matching a <code class="literal">WHERE</code> clause
            quickly.
          </p></li><li class="listitem"><p>
            To eliminate rows from consideration. If there is a choice
            between multiple indexes, MySQL normally uses the index that
            finds the smallest number of rows (the most
            <a class="link" href="glossary.html#glos_selectivity" title="selectivity">selective</a> index).
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444350958176"></a>

            <a class="indexterm" name="idm46444350956688"></a>

            If the table has a multiple-column index, any leftmost
            prefix of the index can be used by the optimizer to look up
            rows. For example, if you have a three-column index on
            <code class="literal">(col1, col2, col3)</code>, you have indexed
            search capabilities on <code class="literal">(col1)</code>,
            <code class="literal">(col1, col2)</code>, and <code class="literal">(col1, col2,
            col3)</code>. For more information, see
            <a class="xref" href="optimization.html#multiple-column-indexes" title="8.3.6 Multiple-Column Indexes">Section 8.3.6, “Multiple-Column Indexes”</a>.
          </p></li><li class="listitem"><p>
            To retrieve rows from other tables when performing joins.
            MySQL can use indexes on columns more efficiently if they
            are declared as the same type and size. In this context,
            <a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">VARCHAR</code></a> and
            <a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">CHAR</code></a> are considered the same
            if they are declared as the same size. For example,
            <code class="literal">VARCHAR(10)</code> and
            <code class="literal">CHAR(10)</code> are the same size, but
            <code class="literal">VARCHAR(10)</code> and
            <code class="literal">CHAR(15)</code> are not.
          </p><p>
            For comparisons between nonbinary string columns, both
            columns should use the same character set. For example,
            comparing a <code class="literal">utf8</code> column with a
            <code class="literal">latin1</code> column precludes use of an index.
          </p><p>
            Comparison of dissimilar columns (comparing a string column
            to a temporal or numeric column, for example) may prevent
            use of indexes if values cannot be compared directly without
            conversion. For a given value such as <code class="literal">1</code>
            in the numeric column, it might compare equal to any number
            of values in the string column such as
            <code class="literal">'1'</code>, <code class="literal">' 1'</code>,
            <code class="literal">'00001'</code>, or <code class="literal">'01.e1'</code>.
            This rules out use of any indexes for the string column.
          </p></li><li class="listitem"><p>
            To find the <a class="link" href="functions.html#function_min"><code class="literal">MIN()</code></a> or
            <a class="link" href="functions.html#function_max"><code class="literal">MAX()</code></a> value for a specific
            indexed column <em class="replaceable"><code>key_col</code></em>. This is
            optimized by a preprocessor that checks whether you are
            using <code class="literal">WHERE <em class="replaceable"><code>key_part_N</code></em> =
            <em class="replaceable"><code>constant</code></em></code> on all key
            parts that occur before <em class="replaceable"><code>key_col</code></em>
            in the index. In this case, MySQL does a single key lookup
            for each <a class="link" href="functions.html#function_min"><code class="literal">MIN()</code></a> or
            <a class="link" href="functions.html#function_max"><code class="literal">MAX()</code></a> expression and replaces
            it with a constant. If all expressions are replaced with
            constants, the query returns at once. For example:
          </p><pre data-lang="sql" class="programlisting">SELECT MIN(<em class="replaceable"><code>key_part2</code></em>),MAX(<em class="replaceable"><code>key_part2</code></em>)
  FROM <em class="replaceable"><code>tbl_name</code></em> WHERE <em class="replaceable"><code>key_part1</code></em>=10;
</pre></li><li class="listitem"><p>
            To sort or group a table if the sorting or grouping is done
            on a leftmost prefix of a usable index (for example,
            <code class="literal">ORDER BY <em class="replaceable"><code>key_part1</code></em>,
            <em class="replaceable"><code>key_part2</code></em></code>). If all key
            parts are followed by <code class="literal">DESC</code>, the key is
            read in reverse order. (Or, if the index is a descending
            index, the key is read in forward order.) See
            <a class="xref" href="optimization.html#order-by-optimization" title="8.2.1.16 ORDER BY Optimization">Section 8.2.1.16, “ORDER BY Optimization”</a>,
            <a class="xref" href="optimization.html#group-by-optimization" title="8.2.1.17 GROUP BY Optimization">Section 8.2.1.17, “GROUP BY Optimization”</a>, and
            <a class="xref" href="optimization.html#descending-indexes" title="8.3.13 Descending Indexes">Section 8.3.13, “Descending Indexes”</a>.
          </p></li><li class="listitem"><p>
            In some cases, a query can be optimized to retrieve values
            without consulting the data rows. (An index that provides
            all the necessary results for a query is called a
            <a class="link" href="glossary.html#glos_covering_index" title="covering index">covering index</a>.)
            If a query uses from a table only columns that are included
            in some index, the selected values can be retrieved from the
            index tree for greater speed:
          </p><pre data-lang="sql" class="programlisting">SELECT <em class="replaceable"><code>key_part3</code></em> FROM <em class="replaceable"><code>tbl_name</code></em>
  WHERE <em class="replaceable"><code>key_part1</code></em>=1
</pre></li></ul>
</div>
<p>
        Indexes are less important for queries on small tables, or big
        tables where report queries process most or all of the rows.
        When a query needs to access most of the rows, reading
        sequentially is faster than working through an index. Sequential
        reads minimize disk seeks, even if not all the rows are needed
        for the query. See <a class="xref" href="optimization.html#table-scan-avoidance" title="8.2.1.23 Avoiding Full Table Scans">Section 8.2.1.23, “Avoiding Full Table Scans”</a> for
        details.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="primary-key-optimization"></a>8.3.2 Primary Key Optimization</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350914688"></a><p>
        The primary key for a table represents the column or set of
        columns that you use in your most vital queries. It has an
        associated index, for fast query performance. Query performance
        benefits from the <code class="literal">NOT NULL</code> optimization,
        because it cannot include any <code class="literal">NULL</code> values.
        With the <code class="literal">InnoDB</code> storage engine, the table
        data is physically organized to do ultra-fast lookups and sorts
        based on the primary key column or columns.
      </p><p>
        If your table is big and important, but does not have an obvious
        column or set of columns to use as a primary key, you might
        create a separate column with auto-increment values to use as
        the primary key. These unique IDs can serve as pointers to
        corresponding rows in other tables when you join tables using
        foreign keys.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="spatial-index-optimization"></a>8.3.3 SPATIAL Index Optimization</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350907808"></a><a class="indexterm" name="idm46444350906320"></a><p>
        MySQL permits creation of <code class="literal">SPATIAL</code> indexes on
        <code class="literal">NOT NULL</code> geometry-valued columns (see
        <a class="xref" href="data-types.html#creating-spatial-indexes" title="11.4.10 Creating Spatial Indexes">Section 11.4.10, “Creating Spatial Indexes”</a>). The optimizer
        checks the <code class="literal">SRID</code> attribute for indexed columns
        to determine which spatial reference system (SRS) to use for
        comparisons, and uses calculations appropriate to the SRS.
        (Prior to MySQL 8.0, the optimizer performs
        comparisons of <code class="literal">SPATIAL</code> index values using
        Cartesian calculations; the results of such operations are
        undefined if the column contains values with non-Cartesian
        SRIDs.)
      </p><p>
        For comparisons to work properly, each column in a
        <code class="literal">SPATIAL</code> index must be SRID-restricted. That
        is, the column definition must include an explicit
        <code class="literal">SRID</code> attribute, and all column values must
        have the same SRID.
      </p><p>
        The optimizer considers <code class="literal">SPATIAL</code> indexes only
        for SRID-restricted columns:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Indexes on columns restricted to a Cartesian SRID enable
            Cartesian bounding box computations.
          </p></li><li class="listitem"><p>
            Indexes on columns restricted to a geographic SRID enable
            geographic bounding box computations.
</p></li></ul>
</div>
<p>
        The optimizer ignores <code class="literal">SPATIAL</code> indexes on
        columns that have no <code class="literal">SRID</code> attribute (and thus
        are not SRID-restricted). MySQL still maintains such indexes, as
        follows:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            They are updated for table modifications
            (<a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>,
            <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a>,
            <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a>, and so forth).
            Updates occur as though the index was Cartesian, even though
            the column might contain a mix of Cartesian and geographical
            values.
          </p></li><li class="listitem"><p>
            They exist only for backward compatibility (for example, the
            ability to perform a dump in MySQL 5.7 and restore in MySQL
            8.0). Because <code class="literal">SPATIAL</code> indexes on columns
            that are not SRID-restricted are of no use to the optimizer,
            each such column should be modified:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                Verify that all values within the column have the same
                SRID. To determine the SRIDs contained in a geometry
                column <em class="replaceable"><code>col_name</code></em>, use the
                following query:
              </p><pre data-lang="sql" class="programlisting">SELECT DISTINCT ST_SRID(<em class="replaceable"><code>col_name</code></em>) FROM <em class="replaceable"><code>tbl_name</code></em>;
</pre><p>
                If the query returns more than one row, the column
                contains a mix of SRIDs. In that case, modify its
                contents so all values have the same SRID.
              </p></li><li class="listitem"><p>
                Redefine the column to have an explicit
                <code class="literal">SRID</code> attribute.
              </p></li><li class="listitem"><p>
                Recreate the <code class="literal">SPATIAL</code> index.
</p></li></ul>
</div>
</li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="foreign-key-optimization"></a>8.3.4 Foreign Key Optimization</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350877184"></a><p>
        If a table has many columns, and you query many different
        combinations of columns, it might be efficient to split the
        less-frequently used data into separate tables with a few
        columns each, and relate them back to the main table by
        duplicating the numeric ID column from the main table. That way,
        each small table can have a primary key for fast lookups of its
        data, and you can query just the set of columns that you need
        using a join operation. Depending on how the data is
        distributed, the queries might perform less I/O and take up less
        cache memory because the relevant columns are packed together on
        disk. (To maximize performance, queries try to read as few data
        blocks as possible from disk; tables with only a few columns can
        fit more rows in each data block.)
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="column-indexes"></a>8.3.5 Column Indexes</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350874032"></a><a class="indexterm" name="idm46444350872544"></a><a class="indexterm" name="idm46444350871056"></a><p>
        The most common type of index involves a single column, storing
        copies of the values from that column in a data structure,
        allowing fast lookups for the rows with the corresponding column
        values. The B-tree data structure lets the index quickly find a
        specific value, a set of values, or a range of values,
        corresponding to operators such as <code class="literal">=</code>,
        <code class="literal">&gt;</code>, <code class="literal">≤</code>,
        <code class="literal">BETWEEN</code>, <code class="literal">IN</code>, and so on, in
        a <code class="literal">WHERE</code> clause.
      </p><p>
        The maximum number of indexes per table and the maximum index
        length is defined per storage engine. See
        <a class="xref" href="innodb-storage-engine.html" title="Chapter 15 The InnoDB Storage Engine">Chapter 15, <i>The InnoDB Storage Engine</i></a>, and
        <a class="xref" href="storage-engines.html" title="Chapter 16 Alternative Storage Engines">Chapter 16, <i>Alternative Storage Engines</i></a>. All storage engines support
        at least 16 indexes per table and a total index length of at
        least 256 bytes. Most storage engines have higher limits.
      </p><p>
        For additional information about column indexes, see
        <a class="xref" href="sql-statements.html#create-index" title="13.1.15 CREATE INDEX Statement">Section 13.1.15, “CREATE INDEX Statement”</a>.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a class="xref" href="optimization.html#column-indexes-prefix" title="Index Prefixes">Index Prefixes</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#column-indexes-fulltext" title="FULLTEXT Indexes">FULLTEXT Indexes</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#column-indexes-spatial" title="Spatial Indexes">Spatial Indexes</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#column-indexes-memory-storage-engine" title="Indexes in the MEMORY Storage Engine">Indexes in the MEMORY Storage Engine</a></p></li></ul>
</div>

<div class="simplesect">

<div class="titlepage">
<div>

<div class="simple">
<h4 class="title"><a name="column-indexes-prefix"></a>Index Prefixes</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350856096"></a><a class="indexterm" name="idm46444350854608"></a><a class="indexterm" name="idm46444350853120"></a><a class="indexterm" name="idm46444350851632"></a><a class="indexterm" name="idm46444350850144"></a><p>
          With
          <code class="literal"><em class="replaceable"><code>col_name</code></em>(<em class="replaceable"><code>N</code></em>)</code>
          syntax in an index specification for a string column, you can
          create an index that uses only the first
          <em class="replaceable"><code>N</code></em> characters of the column.
          Indexing only a prefix of column values in this way can make
          the index file much smaller. When you index a
          <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a> or
          <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">TEXT</code></a> column, you
          <span class="emphasis"><em>must</em></span> specify a prefix length for the
          index. For example:
        </p><pre data-lang="sql" class="programlisting">CREATE TABLE test (blob_col BLOB, INDEX(blob_col(10)));</pre><p>
          Prefixes can be up to 767 bytes long for
          <code class="literal">InnoDB</code> tables that use the
          <code class="literal"><a class="link" href="glossary.html#glos_redundant_row_format" title="redundant row format">REDUNDANT</a></code>
          or
          <code class="literal"><a class="link" href="glossary.html#glos_compact_row_format" title="compact row format">COMPACT</a></code>
          row format. The prefix length limit is 3072 bytes for
          <code class="literal">InnoDB</code> tables that use the
          <code class="literal"><a class="link" href="glossary.html#glos_dynamic_row_format" title="dynamic row format">DYNAMIC</a></code>
          or
          <code class="literal"><a class="link" href="glossary.html#glos_compressed_row_format" title="compressed row format">COMPRESSED</a></code>
          row format. For MyISAM tables, the prefix length limit is 1000
          bytes.
</p>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
            Prefix limits are measured in bytes, whereas the prefix
            length in <a class="link" href="sql-statements.html#create-table" title="13.1.20 CREATE TABLE Statement"><code class="literal">CREATE TABLE</code></a>,
            <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER TABLE</code></a>, and
            <a class="link" href="sql-statements.html#create-index" title="13.1.15 CREATE INDEX Statement"><code class="literal">CREATE INDEX</code></a> statements is
            interpreted as number of characters for nonbinary string
            types (<a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">CHAR</code></a>,
            <a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">VARCHAR</code></a>,
            <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">TEXT</code></a>) and number of bytes for
            binary string types (<a class="link" href="data-types.html#binary-varbinary" title="11.3.3 The BINARY and VARBINARY Types"><code class="literal">BINARY</code></a>,
            <a class="link" href="data-types.html#binary-varbinary" title="11.3.3 The BINARY and VARBINARY Types"><code class="literal">VARBINARY</code></a>,
            <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a>). Take this into account
            when specifying a prefix length for a nonbinary string
            column that uses a multibyte character set.
</p>
</div>
<p>
          If a search term exceeds the index prefix length, the index is
          used to exclude non-matching rows, and the remaining rows are
          examined for possible matches.
        </p><p>
          For additional information about index prefixes, see
          <a class="xref" href="sql-statements.html#create-index" title="13.1.15 CREATE INDEX Statement">Section 13.1.15, “CREATE INDEX Statement”</a>.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="column-indexes-fulltext"></a>FULLTEXT Indexes</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350819472"></a><a class="indexterm" name="idm46444350817984"></a><p>
          <code class="literal">FULLTEXT</code> indexes are used for full-text
          searches. Only the <a class="link" href="innodb-storage-engine.html" title="Chapter 15 The InnoDB Storage Engine"><code class="literal">InnoDB</code></a> and
          <a class="link" href="storage-engines.html#myisam-storage-engine" title="16.2 The MyISAM Storage Engine"><code class="literal">MyISAM</code></a> storage engines support
          <code class="literal">FULLTEXT</code> indexes and only for
          <a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">CHAR</code></a>,
          <a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">VARCHAR</code></a>, and
          <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">TEXT</code></a> columns. Indexing always
          takes place over the entire column and column prefix indexing
          is not supported. For details, see
          <a class="xref" href="functions.html#fulltext-search" title="12.9 Full-Text Search Functions">Section 12.9, “Full-Text Search Functions”</a>.
        </p><p>
          Optimizations are applied to certain kinds of
          <code class="literal">FULLTEXT</code> queries against single
          <code class="literal">InnoDB</code> tables. Queries with these
          characteristics are particularly efficient:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <code class="literal">FULLTEXT</code> queries that only return the
              document ID, or the document ID and the search rank.
            </p></li><li class="listitem"><p>
              <code class="literal">FULLTEXT</code> queries that sort the matching
              rows in descending order of score and apply a
              <code class="literal">LIMIT</code> clause to take the top N matching
              rows. For this optimization to apply, there must be no
              <code class="literal">WHERE</code> clauses and only a single
              <code class="literal">ORDER BY</code> clause in descending order.
            </p></li><li class="listitem"><p>
              <code class="literal">FULLTEXT</code> queries that retrieve only the
              <code class="literal">COUNT(*)</code> value of rows matching a
              search term, with no additional <code class="literal">WHERE</code>
              clauses. Code the <code class="literal">WHERE</code> clause as
              <code class="literal">WHERE MATCH(<em class="replaceable"><code>text</code></em>)
              AGAINST
              ('<em class="replaceable"><code>other_text</code></em>')</code>,
              without any <code class="literal">&gt; 0</code> comparison operator.
</p></li></ul>
</div>
<p>
          For queries that contain full-text expressions, MySQL
          evaluates those expressions during the optimization phase of
          query execution. The optimizer does not just look at full-text
          expressions and make estimates, it actually evaluates them in
          the process of developing an execution plan.
        </p><p>
          An implication of this behavior is that
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> for full-text queries
          is typically slower than for non-full-text queries for which
          no expression evaluation occurs during the optimization phase.
        </p><p>
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> for full-text queries
          may show <code class="literal">Select tables optimized away</code> in
          the <code class="literal">Extra</code> column due to matching occurring
          during optimization; in this case, no table access need occur
          during later execution.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="column-indexes-spatial"></a>Spatial Indexes</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350786352"></a><a class="indexterm" name="idm46444350784864"></a><p>
          You can create indexes on spatial data types.
          <code class="literal">MyISAM</code> and <code class="literal">InnoDB</code>
          support R-tree indexes on spatial types. Other storage engines
          use B-trees for indexing spatial types (except for
          <code class="literal">ARCHIVE</code>, which does not support spatial
          type indexing).
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="column-indexes-memory-storage-engine"></a>Indexes in the MEMORY Storage Engine</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350779296"></a><a class="indexterm" name="idm46444350777808"></a><p>
          The <code class="literal">MEMORY</code> storage engine uses
          <code class="literal">HASH</code> indexes by default, but also supports
          <code class="literal">BTREE</code> indexes.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="multiple-column-indexes"></a>8.3.6 Multiple-Column Indexes</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350772336"></a><a class="indexterm" name="idm46444350771264"></a><a class="indexterm" name="idm46444350769776"></a><p>
        MySQL can create composite indexes (that is, indexes on multiple
        columns). An index may consist of up to 16 columns. For certain
        data types, you can index a prefix of the column (see
        <a class="xref" href="optimization.html#column-indexes" title="8.3.5 Column Indexes">Section 8.3.5, “Column Indexes”</a>).
      </p><p>
        MySQL can use multiple-column indexes for queries that test all
        the columns in the index, or queries that test just the first
        column, the first two columns, the first three columns, and so
        on. If you specify the columns in the right order in the index
        definition, a single composite index can speed up several kinds
        of queries on the same table.
      </p><p>
        A multiple-column index can be considered a sorted array, the
        rows of which contain values that are created by concatenating
        the values of the indexed columns.
</p>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
          As an alternative to a composite index, you can introduce a
          column that is <span class="quote">“<span class="quote">hashed</span>”</span> based on information from
          other columns. If this column is short, reasonably unique, and
          indexed, it might be faster than a <span class="quote">“<span class="quote">wide</span>”</span> index
          on many columns. In MySQL, it is very easy to use this extra
          column:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM <em class="replaceable"><code>tbl_name</code></em>
  WHERE <em class="replaceable"><code>hash_col</code></em>=MD5(CONCAT(<em class="replaceable"><code>val1</code></em>,<em class="replaceable"><code>val2</code></em>))
  AND <em class="replaceable"><code>col1</code></em>=<em class="replaceable"><code>val1</code></em> AND <em class="replaceable"><code>col2</code></em>=<em class="replaceable"><code>val2</code></em>;
</pre>
</div>
<p>
        Suppose that a table has the following specification:
      </p><pre data-lang="sql" class="programlisting">CREATE TABLE test (
    id         INT NOT NULL,
    last_name  CHAR(30) NOT NULL,
    first_name CHAR(30) NOT NULL,
    PRIMARY KEY (id),
    INDEX name (last_name,first_name)
);</pre><p>
        The <code class="literal">name</code> index is an index over the
        <code class="literal">last_name</code> and <code class="literal">first_name</code>
        columns. The index can be used for lookups in queries that
        specify values in a known range for combinations of
        <code class="literal">last_name</code> and <code class="literal">first_name</code>
        values. It can also be used for queries that specify just a
        <code class="literal">last_name</code> value because that column is a
        leftmost prefix of the index (as described later in this
        section). Therefore, the <code class="literal">name</code> index is used
        for lookups in the following queries:
      </p><pre data-lang="sql" class="programlisting">SELECT * FROM test WHERE last_name='Jones';

SELECT * FROM test
  WHERE last_name='Jones' AND first_name='John';

SELECT * FROM test
  WHERE last_name='Jones'
  AND (first_name='John' OR first_name='Jon');

SELECT * FROM test
  WHERE last_name='Jones'
  AND first_name &gt;='M' AND first_name &lt; 'N';</pre><p>
        However, the <code class="literal">name</code> index is
        <span class="emphasis"><em>not</em></span> used for lookups in the following
        queries:
      </p><pre data-lang="sql" class="programlisting">SELECT * FROM test WHERE first_name='John';

SELECT * FROM test
  WHERE last_name='Jones' OR first_name='John';</pre><p>
        Suppose that you issue the following
        <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statement:
      </p><pre data-lang="sql" class="programlisting">SELECT * FROM <em class="replaceable"><code>tbl_name</code></em>
  WHERE col1=<em class="replaceable"><code>val1</code></em> AND col2=<em class="replaceable"><code>val2</code></em>;
</pre><p>
        If a multiple-column index exists on <code class="literal">col1</code> and
        <code class="literal">col2</code>, the appropriate rows can be fetched
        directly. If separate single-column indexes exist on
        <code class="literal">col1</code> and <code class="literal">col2</code>, the
        optimizer attempts to use the Index Merge optimization (see
        <a class="xref" href="optimization.html#index-merge-optimization" title="8.2.1.3 Index Merge Optimization">Section 8.2.1.3, “Index Merge Optimization”</a>), or attempts to find
        the most restrictive index by deciding which index excludes more
        rows and using that index to fetch the rows.
      </p><a class="indexterm" name="idm46444350739392"></a><a class="indexterm" name="idm46444350737904"></a><p>
        If the table has a multiple-column index, any leftmost prefix of
        the index can be used by the optimizer to look up rows. For
        example, if you have a three-column index on <code class="literal">(col1,
        col2, col3)</code>, you have indexed search capabilities on
        <code class="literal">(col1)</code>, <code class="literal">(col1, col2)</code>, and
        <code class="literal">(col1, col2, col3)</code>.
      </p><p>
        MySQL cannot use the index to perform lookups if the columns do
        not form a leftmost prefix of the index. Suppose that you have
        the <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements shown here:
      </p><pre data-lang="sql" class="programlisting">SELECT * FROM <em class="replaceable"><code>tbl_name</code></em> WHERE col1=<em class="replaceable"><code>val1</code></em>;
SELECT * FROM <em class="replaceable"><code>tbl_name</code></em> WHERE col1=<em class="replaceable"><code>val1</code></em> AND col2=<em class="replaceable"><code>val2</code></em>;

SELECT * FROM <em class="replaceable"><code>tbl_name</code></em> WHERE col2=<em class="replaceable"><code>val2</code></em>;
SELECT * FROM <em class="replaceable"><code>tbl_name</code></em> WHERE col2=<em class="replaceable"><code>val2</code></em> AND col3=<em class="replaceable"><code>val3</code></em>;
</pre><p>
        If an index exists on <code class="literal">(col1, col2, col3)</code>,
        only the first two queries use the index. The third and fourth
        queries do involve indexed columns, but do not use an index to
        perform lookups because <code class="literal">(col2)</code> and
        <code class="literal">(col2, col3)</code> are not leftmost prefixes of
        <code class="literal">(col1, col2, col3)</code>.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="verifying-index-usage"></a>8.3.7 Verifying Index Usage</h3>

</div>

</div>

</div>
<p>
        Always check whether all your queries really use the indexes
        that you have created in the tables. Use the
        <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> statement, as described
        in <a class="xref" href="optimization.html#using-explain" title="8.8.1 Optimizing Queries with EXPLAIN">Section 8.8.1, “Optimizing Queries with EXPLAIN”</a>.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="index-statistics"></a>8.3.8 InnoDB and MyISAM Index Statistics Collection</h3>

</div>

</div>

</div>
<p>
        Storage engines collect statistics about tables for use by the
        optimizer. Table statistics are based on value groups, where a
        value group is a set of rows with the same key prefix value. For
        optimizer purposes, an important statistic is the average value
        group size.
      </p><p>
        MySQL uses the average value group size in the following ways:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            To estimate how many rows must be read for each
            <a class="link" href="optimization.html#jointype_ref"><code class="literal">ref</code></a> access
          </p></li><li class="listitem"><p>
            To estimate how many rows a partial join will produce; that
            is, the number of rows that an operation of this form will
            produce:
          </p><pre data-lang="sql" class="programlisting">(...) JOIN <em class="replaceable"><code>tbl_name</code></em> ON <em class="replaceable"><code>tbl_name</code></em>.<em class="replaceable"><code>key</code></em> = <em class="replaceable"><code>expr</code></em>
</pre></li></ul>
</div>
<p>
        As the average value group size for an index increases, the
        index is less useful for those two purposes because the average
        number of rows per lookup increases: For the index to be good
        for optimization purposes, it is best that each index value
        target a small number of rows in the table. When a given index
        value yields a large number of rows, the index is less useful
        and MySQL is less likely to use it.
      </p><p>
        The average value group size is related to table cardinality,
        which is the number of value groups. The
        <a class="link" href="sql-statements.html#show-index" title="13.7.7.22 SHOW INDEX Statement"><code class="literal">SHOW INDEX</code></a> statement displays a
        cardinality value based on <em class="replaceable"><code>N/S</code></em>, where
        <em class="replaceable"><code>N</code></em> is the number of rows in the table
        and <em class="replaceable"><code>S</code></em> is the average value group
        size. That ratio yields an approximate number of value groups in
        the table.
      </p><p>
        For a join based on the <code class="literal">&lt;=&gt;</code> comparison
        operator, <code class="literal">NULL</code> is not treated differently
        from any other value: <code class="literal">NULL &lt;=&gt; NULL</code>,
        just as <code class="literal"><em class="replaceable"><code>N</code></em> &lt;=&gt;
        <em class="replaceable"><code>N</code></em></code> for any other
        <em class="replaceable"><code>N</code></em>.
      </p><p>
        However, for a join based on the <code class="literal">=</code> operator,
        <code class="literal">NULL</code> is different from
        non-<code class="literal">NULL</code> values:
        <code class="literal"><em class="replaceable"><code>expr1</code></em> =
        <em class="replaceable"><code>expr2</code></em></code> is not true when
        <em class="replaceable"><code>expr1</code></em> or
        <em class="replaceable"><code>expr2</code></em> (or both) are
        <code class="literal">NULL</code>. This affects
        <a class="link" href="optimization.html#jointype_ref"><code class="literal">ref</code></a> accesses for comparisons
        of the form <code class="literal"><em class="replaceable"><code>tbl_name.key</code></em> =
        <em class="replaceable"><code>expr</code></em></code>: MySQL will not access
        the table if the current value of
        <em class="replaceable"><code>expr</code></em> is <code class="literal">NULL</code>,
        because the comparison cannot be true.
      </p><p>
        For <code class="literal">=</code> comparisons, it does not matter how
        many <code class="literal">NULL</code> values are in the table. For
        optimization purposes, the relevant value is the average size of
        the non-<code class="literal">NULL</code> value groups. However, MySQL
        does not currently enable that average size to be collected or
        used.
      </p><p>
        For <code class="literal">InnoDB</code> and <code class="literal">MyISAM</code>
        tables, you have some control over collection of table
        statistics by means of the
        <a class="link" href="innodb-storage-engine.html#sysvar_innodb_stats_method"><code class="literal">innodb_stats_method</code></a> and
        <a class="link" href="server-administration.html#sysvar_myisam_stats_method"><code class="literal">myisam_stats_method</code></a> system
        variables, respectively. These variables have three possible
        values, which differ as follows:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            When the variable is set to <code class="literal">nulls_equal</code>,
            all <code class="literal">NULL</code> values are treated as identical
            (that is, they all form a single value group).
          </p><p>
            If the <code class="literal">NULL</code> value group size is much
            higher than the average non-<code class="literal">NULL</code> value
            group size, this method skews the average value group size
            upward. This makes index appear to the optimizer to be less
            useful than it really is for joins that look for
            non-<code class="literal">NULL</code> values. Consequently, the
            <code class="literal">nulls_equal</code> method may cause the
            optimizer not to use the index for
            <a class="link" href="optimization.html#jointype_ref"><code class="literal">ref</code></a> accesses when it
            should.
          </p></li><li class="listitem"><p>
            When the variable is set to
            <code class="literal">nulls_unequal</code>, <code class="literal">NULL</code>
            values are not considered the same. Instead, each
            <code class="literal">NULL</code> value forms a separate value group
            of size 1.
          </p><p>
            If you have many <code class="literal">NULL</code> values, this method
            skews the average value group size downward. If the average
            non-<code class="literal">NULL</code> value group size is large,
            counting <code class="literal">NULL</code> values each as a group of
            size 1 causes the optimizer to overestimate the value of the
            index for joins that look for non-<code class="literal">NULL</code>
            values. Consequently, the <code class="literal">nulls_unequal</code>
            method may cause the optimizer to use this index for
            <a class="link" href="optimization.html#jointype_ref"><code class="literal">ref</code></a> lookups when other
            methods may be better.
          </p></li><li class="listitem"><p>
            When the variable is set to
            <code class="literal">nulls_ignored</code>, <code class="literal">NULL</code>
            values are ignored.
</p></li></ul>
</div>
<p>
        If you tend to use many joins that use
        <code class="literal">&lt;=&gt;</code> rather than <code class="literal">=</code>,
        <code class="literal">NULL</code> values are not special in comparisons
        and one <code class="literal">NULL</code> is equal to another. In this
        case, <code class="literal">nulls_equal</code> is the appropriate
        statistics method.
      </p><p>
        The <a class="link" href="innodb-storage-engine.html#sysvar_innodb_stats_method"><code class="literal">innodb_stats_method</code></a> system
        variable has a global value; the
        <a class="link" href="server-administration.html#sysvar_myisam_stats_method"><code class="literal">myisam_stats_method</code></a> system
        variable has both global and session values. Setting the global
        value affects statistics collection for tables from the
        corresponding storage engine. Setting the session value affects
        statistics collection only for the current client connection.
        This means that you can force a table's statistics to be
        regenerated with a given method without affecting other clients
        by setting the session value of
        <a class="link" href="server-administration.html#sysvar_myisam_stats_method"><code class="literal">myisam_stats_method</code></a>.
      </p><p>
        To regenerate <code class="literal">MyISAM</code> table statistics, you
        can use any of the following methods:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Execute <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk
            --stats_method=<em class="replaceable"><code>method_name</code></em>
            --analyze</strong></span></a>
          </p></li><li class="listitem"><p>
            Change the table to cause its statistics to go out of date
            (for example, insert a row and then delete it), and then set
            <a class="link" href="server-administration.html#sysvar_myisam_stats_method"><code class="literal">myisam_stats_method</code></a> and
            issue an <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE TABLE</code></a>
            statement
</p></li></ul>
</div>
<p>
        Some caveats regarding the use of
        <a class="link" href="innodb-storage-engine.html#sysvar_innodb_stats_method"><code class="literal">innodb_stats_method</code></a> and
        <a class="link" href="server-administration.html#sysvar_myisam_stats_method"><code class="literal">myisam_stats_method</code></a>:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            You can force table statistics to be collected explicitly,
            as just described. However, MySQL may also collect
            statistics automatically. For example, if during the course
            of executing statements for a table, some of those
            statements modify the table, MySQL may collect statistics.
            (This may occur for bulk inserts or deletes, or some
            <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER TABLE</code></a> statements, for
            example.) If this happens, the statistics are collected
            using whatever value
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_stats_method"><code class="literal">innodb_stats_method</code></a> or
            <a class="link" href="server-administration.html#sysvar_myisam_stats_method"><code class="literal">myisam_stats_method</code></a> has at
            the time. Thus, if you collect statistics using one method,
            but the system variable is set to the other method when a
            table's statistics are collected automatically later,
            the other method will be used.
          </p></li><li class="listitem"><p>
            There is no way to tell which method was used to generate
            statistics for a given table.
          </p></li><li class="listitem"><p>
            These variables apply only to <code class="literal">InnoDB</code> and
            <code class="literal">MyISAM</code> tables. Other storage engines have
            only one method for collecting table statistics. Usually it
            is closer to the <code class="literal">nulls_equal</code> method.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="index-btree-hash"></a>8.3.9 Comparison of B-Tree and Hash Indexes</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350636480"></a><a class="indexterm" name="idm46444350635440"></a><p>
        Understanding the B-tree and hash data structures can help
        predict how different queries perform on different storage
        engines that use these data structures in their indexes,
        particularly for the <code class="literal">MEMORY</code> storage engine
        that lets you choose B-tree or hash indexes.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a class="xref" href="optimization.html#btree-index-characteristics" title="B-Tree Index Characteristics">B-Tree Index Characteristics</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#hash-index-characteristics" title="Hash Index Characteristics">Hash Index Characteristics</a></p></li></ul>
</div>

<div class="simplesect">

<div class="titlepage">
<div>

<div class="simple">
<h4 class="title"><a name="btree-index-characteristics"></a>B-Tree Index Characteristics</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350629520"></a><a class="indexterm" name="idm46444350628032"></a><a class="indexterm" name="idm46444350626544"></a><a class="indexterm" name="idm46444350625056"></a><p>
          A B-tree index can be used for column comparisons in
          expressions that use the
          <a class="link" href="functions.html#operator_equal"><code class="literal">=</code></a>,
          <a class="link" href="functions.html#operator_greater-than"><code class="literal">&gt;</code></a>,
          <a class="link" href="functions.html#operator_greater-than-or-equal"><code class="literal">&gt;=</code></a>,
          <a class="link" href="functions.html#operator_less-than"><code class="literal">&lt;</code></a>,
          <a class="link" href="functions.html#operator_less-than-or-equal"><code class="literal">&lt;=</code></a>,
          or <a class="link" href="functions.html#operator_between"><code class="literal">BETWEEN</code></a> operators. The index
          also can be used for <a class="link" href="functions.html#operator_like"><code class="literal">LIKE</code></a>
          comparisons if the argument to
          <a class="link" href="functions.html#operator_like"><code class="literal">LIKE</code></a> is a constant string that
          does not start with a wildcard character. For example, the
          following <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements use
          indexes:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM <em class="replaceable"><code>tbl_name</code></em> WHERE <em class="replaceable"><code>key_col</code></em> LIKE 'Patrick%';
SELECT * FROM <em class="replaceable"><code>tbl_name</code></em> WHERE <em class="replaceable"><code>key_col</code></em> LIKE 'Pat%_ck%';
</pre><p>
          In the first statement, only rows with <code class="literal">'Patrick'
          &lt;= <em class="replaceable"><code>key_col</code></em> &lt;
          'Patricl'</code> are considered. In the second statement,
          only rows with <code class="literal">'Pat' &lt;=
          <em class="replaceable"><code>key_col</code></em> &lt; 'Pau'</code> are
          considered.
        </p><p>
          The following <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements
          do not use indexes:
        </p><pre data-lang="sql" class="programlisting">SELECT * FROM <em class="replaceable"><code>tbl_name</code></em> WHERE <em class="replaceable"><code>key_col</code></em> LIKE '%Patrick%';
SELECT * FROM <em class="replaceable"><code>tbl_name</code></em> WHERE <em class="replaceable"><code>key_col</code></em> LIKE <em class="replaceable"><code>other_col</code></em>;
</pre><p>
          In the first statement, the <a class="link" href="functions.html#operator_like"><code class="literal">LIKE</code></a>
          value begins with a wildcard character. In the second
          statement, the <a class="link" href="functions.html#operator_like"><code class="literal">LIKE</code></a> value is not
          a constant.
        </p><p>
          If you use <code class="literal">... LIKE
          '%<em class="replaceable"><code>string</code></em>%'</code> and
          <em class="replaceable"><code>string</code></em> is longer than three
          characters, MySQL uses the <span class="firstterm">Turbo
          Boyer-Moore algorithm</span> to initialize the pattern for
          the string and then uses this pattern to perform the search
          more quickly.
        </p><a class="indexterm" name="idm46444350595264"></a><a class="indexterm" name="idm46444350593776"></a><p>
          A search using <code class="literal"><em class="replaceable"><code>col_name</code></em> IS
          NULL</code> employs indexes if
          <em class="replaceable"><code>col_name</code></em> is indexed.
        </p><p>
          Any index that does not span all
          <a class="link" href="functions.html#operator_and"><code class="literal">AND</code></a> levels in the
          <code class="literal">WHERE</code> clause is not used to optimize the
          query. In other words, to be able to use an index, a prefix of
          the index must be used in every
          <a class="link" href="functions.html#operator_and"><code class="literal">AND</code></a> group.
        </p><p>
          The following <code class="literal">WHERE</code> clauses use indexes:
        </p><pre data-lang="sql" class="programlisting">... WHERE <em class="replaceable"><code>index_part1</code></em>=1 AND <em class="replaceable"><code>index_part2</code></em>=2 AND <em class="replaceable"><code>other_column</code></em>=3

    /* <em class="replaceable"><code>index</code></em> = 1 OR <em class="replaceable"><code>index</code></em> = 2 */
... WHERE <em class="replaceable"><code>index</code></em>=1 OR A=10 AND <em class="replaceable"><code>index</code></em>=2

    /* optimized like "<em class="replaceable"><code>index_part1</code></em>='hello'" */
... WHERE <em class="replaceable"><code>index_part1</code></em>='hello' AND <em class="replaceable"><code>index_part3</code></em>=5

    /* Can use index on <em class="replaceable"><code>index1</code></em> but not on <em class="replaceable"><code>index2</code></em> or <em class="replaceable"><code>index3</code></em> */
... WHERE <em class="replaceable"><code>index1</code></em>=1 AND <em class="replaceable"><code>index2</code></em>=2 OR <em class="replaceable"><code>index1</code></em>=3 AND <em class="replaceable"><code>index3</code></em>=3;
</pre><p>
          These <code class="literal">WHERE</code> clauses do
          <span class="emphasis"><em>not</em></span> use indexes:
        </p><pre data-lang="sql" class="programlisting">    /* <em class="replaceable"><code>index_part1</code></em> is not used */
... WHERE <em class="replaceable"><code>index_part2</code></em>=1 AND <em class="replaceable"><code>index_part3</code></em>=2

    /*  Index is not used in both parts of the WHERE clause  */
... WHERE <em class="replaceable"><code>index</code></em>=1 OR A=10

    /* No index spans all rows  */
... WHERE <em class="replaceable"><code>index_part1</code></em>=1 OR <em class="replaceable"><code>index_part2</code></em>=10
</pre><p>
          Sometimes MySQL does not use an index, even if one is
          available. One circumstance under which this occurs is when
          the optimizer estimates that using the index would require
          MySQL to access a very large percentage of the rows in the
          table. (In this case, a table scan is likely to be much faster
          because it requires fewer seeks.) However, if such a query
          uses <code class="literal">LIMIT</code> to retrieve only some of the
          rows, MySQL uses an index anyway, because it can much more
          quickly find the few rows to return in the result.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="hash-index-characteristics"></a>Hash Index Characteristics</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350568688"></a><p>
          Hash indexes have somewhat different characteristics from
          those just discussed:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              They are used only for equality comparisons that use the
              <code class="literal">=</code> or <code class="literal">&lt;=&gt;</code>
              operators (but are <span class="emphasis"><em>very</em></span> fast). They
              are not used for comparison operators such as
              <code class="literal">&lt;</code> that find a range of values.
              Systems that rely on this type of single-value lookup are
              known as <span class="quote">“<span class="quote">key-value stores</span>”</span>; to use MySQL for
              such applications, use hash indexes wherever possible.
            </p></li><li class="listitem"><p>
              The optimizer cannot use a hash index to speed up
              <code class="literal">ORDER BY</code> operations. (This type of
              index cannot be used to search for the next entry in
              order.)
            </p></li><li class="listitem"><p>
              MySQL cannot determine approximately how many rows there
              are between two values (this is used by the range
              optimizer to decide which index to use). This may affect
              some queries if you change a <code class="literal">MyISAM</code> or
              <code class="literal">InnoDB</code> table to a hash-indexed
              <code class="literal">MEMORY</code> table.
            </p></li><li class="listitem"><p>
              Only whole keys can be used to search for a row. (With a
              B-tree index, any leftmost prefix of the key can be used
              to find rows.)
</p></li></ul>
</div>

</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="index-extensions"></a>8.3.10 Use of Index Extensions</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350555264"></a><p>
        <a class="link" href="innodb-storage-engine.html" title="Chapter 15 The InnoDB Storage Engine"><code class="literal">InnoDB</code></a> automatically extends each
        secondary index by appending the primary key columns to it.
        Consider this table definition:
      </p><pre data-lang="sql" class="programlisting">CREATE TABLE t1 (
  i1 INT NOT NULL DEFAULT 0,
  i2 INT NOT NULL DEFAULT 0,
  d DATE DEFAULT NULL,
  PRIMARY KEY (i1, i2),
  INDEX k_d (d)
) ENGINE = InnoDB;</pre><p>
        This table defines the primary key on columns <code class="literal">(i1,
        i2)</code>. It also defines a secondary index
        <code class="literal">k_d</code> on column <code class="literal">(d)</code>, but
        internally <code class="literal">InnoDB</code> extends this index and
        treats it as columns <code class="literal">(d, i1, i2)</code>.
      </p><p>
        The optimizer takes into account the primary key columns of the
        extended secondary index when determining how and whether to use
        that index. This can result in more efficient query execution
        plans and better performance.
      </p><p>
        The optimizer can use extended secondary indexes for
        <code class="literal">ref</code>, <code class="literal">range</code>, and
        <code class="literal">index_merge</code> index access, for Loose Index
        Scan access, for join and sorting optimization, and for
        <a class="link" href="functions.html#function_min"><code class="literal">MIN()</code></a>/<a class="link" href="functions.html#function_max"><code class="literal">MAX()</code></a>
        optimization.
      </p><p>
        The following example shows how execution plans are affected by
        whether the optimizer uses extended secondary indexes. Suppose
        that <code class="literal">t1</code> is populated with these rows:
      </p><pre data-lang="sql" class="programlisting">INSERT INTO t1 VALUES
(1, 1, '1998-01-01'), (1, 2, '1999-01-01'),
(1, 3, '2000-01-01'), (1, 4, '2001-01-01'),
(1, 5, '2002-01-01'), (2, 1, '1998-01-01'),
(2, 2, '1999-01-01'), (2, 3, '2000-01-01'),
(2, 4, '2001-01-01'), (2, 5, '2002-01-01'),
(3, 1, '1998-01-01'), (3, 2, '1999-01-01'),
(3, 3, '2000-01-01'), (3, 4, '2001-01-01'),
(3, 5, '2002-01-01'), (4, 1, '1998-01-01'),
(4, 2, '1999-01-01'), (4, 3, '2000-01-01'),
(4, 4, '2001-01-01'), (4, 5, '2002-01-01'),
(5, 1, '1998-01-01'), (5, 2, '1999-01-01'),
(5, 3, '2000-01-01'), (5, 4, '2001-01-01'),
(5, 5, '2002-01-01');</pre><p>
        Now consider this query:
      </p><pre data-lang="sql" class="programlisting">EXPLAIN SELECT COUNT(*) FROM t1 WHERE i1 = 3 AND d = '2000-01-01'</pre><p>
        The execution plan depends on whether the extended index is
        used.
      </p><p>
        When the optimizer does not consider index extensions, it treats
        the index <code class="literal">k_d</code> as only <code class="literal">(d)</code>.
        <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> for the query produces
        this result:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN SELECT COUNT(*) FROM t1 WHERE i1 = 3 AND d = '2000-01-01'\G</code></strong>
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: t1
         type: ref
possible_keys: PRIMARY,k_d
          key: k_d
      key_len: 4
          ref: const
         rows: 5
        Extra: Using where; Using index
</pre><p>
        When the optimizer takes index extensions into account, it
        treats <code class="literal">k_d</code> as <code class="literal">(d, i1, i2)</code>.
        In this case, it can use the leftmost index prefix <code class="literal">(d,
        i1)</code> to produce a better execution plan:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN SELECT COUNT(*) FROM t1 WHERE i1 = 3 AND d = '2000-01-01'\G</code></strong>
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: t1
         type: ref
possible_keys: PRIMARY,k_d
          key: k_d
      key_len: 8
          ref: const,const
         rows: 1
        Extra: Using index
</pre><p>
        In both cases, <code class="literal">key</code> indicates that the
        optimizer will use secondary index <code class="literal">k_d</code> but
        the <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output shows these
        improvements from using the extended index:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            <code class="literal">key_len</code> goes from 4 bytes to 8 bytes,
            indicating that key lookups use columns <code class="literal">d</code>
            and <code class="literal">i1</code>, not just <code class="literal">d</code>.
          </p></li><li class="listitem"><p>
            The <code class="literal">ref</code> value changes from
            <code class="literal">const</code> to <code class="literal">const,const</code>
            because the key lookup uses two key parts, not one.
          </p></li><li class="listitem"><p>
            The <code class="literal">rows</code> count decreases from 5 to 1,
            indicating that <code class="literal">InnoDB</code> should need to
            examine fewer rows to produce the result.
          </p></li><li class="listitem"><p>
            The <code class="literal">Extra</code> value changes from
            <code class="literal">Using where; Using index</code> to
            <code class="literal">Using index</code>. This means that rows can be
            read using only the index, without consulting columns in the
            data row.
</p></li></ul>
</div>
<p>
        Differences in optimizer behavior for use of extended indexes
        can also be seen with <a class="link" href="sql-statements.html#show-status" title="13.7.7.35 SHOW STATUS Statement"><code class="literal">SHOW
        STATUS</code></a>:
      </p><pre data-lang="sql" class="programlisting">FLUSH TABLE t1;
FLUSH STATUS;
SELECT COUNT(*) FROM t1 WHERE i1 = 3 AND d = '2000-01-01';
SHOW STATUS LIKE 'handler_read%'</pre><p>
        The preceding statements include <a class="link" href="sql-statements.html#flush-tables"><code class="literal">FLUSH
        TABLES</code></a> and <a class="link" href="sql-statements.html#flush-status"><code class="literal">FLUSH STATUS</code></a>
        to flush the table cache and clear the status counters.
      </p><p>
        Without index extensions, <a class="link" href="sql-statements.html#show-status" title="13.7.7.35 SHOW STATUS Statement"><code class="literal">SHOW
        STATUS</code></a> produces this result:
      </p><pre data-lang="none" class="programlisting">+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Handler_read_first    | 0     |
| Handler_read_key      | 1     |
| Handler_read_last     | 0     |
| Handler_read_next     | 5     |
| Handler_read_prev     | 0     |
| Handler_read_rnd      | 0     |
| Handler_read_rnd_next | 0     |
+-----------------------+-------+</pre><p>
        With index extensions, <a class="link" href="sql-statements.html#show-status" title="13.7.7.35 SHOW STATUS Statement"><code class="literal">SHOW
        STATUS</code></a> produces this result. The
        <a class="link" href="server-administration.html#statvar_Handler_read_next"><code class="literal">Handler_read_next</code></a> value
        decreases from 5 to 1, indicating more efficient use of the
        index:
      </p><pre data-lang="none" class="programlisting">+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Handler_read_first    | 0     |
| Handler_read_key      | 1     |
| Handler_read_last     | 0     |
| Handler_read_next     | 1     |
| Handler_read_prev     | 0     |
| Handler_read_rnd      | 0     |
| Handler_read_rnd_next | 0     |
+-----------------------+-------+</pre><p>
        The <code class="literal">use_index_extensions</code> flag of the
        <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
        variable permits control over whether the optimizer takes the
        primary key columns into account when determining how to use an
        <code class="literal">InnoDB</code> table's secondary indexes. By
        default, <code class="literal">use_index_extensions</code> is enabled. To
        check whether disabling use of index extensions will improve
        performance, use this statement:
      </p><pre data-lang="sql" class="programlisting">SET optimizer_switch = 'use_index_extensions=off';</pre><p>
        Use of index extensions by the optimizer is subject to the usual
        limits on the number of key parts in an index (16) and the
        maximum key length (3072 bytes).
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="generated-column-index-optimizations"></a>8.3.11 Optimizer Use of Generated Column Indexes</h3>

</div>

</div>

</div>
<p>
        MySQL supports indexes on generated columns. For example:
      </p><pre data-lang="sql" class="programlisting">CREATE TABLE t1 (f1 INT, gc INT AS (f1 + 1) STORED, INDEX (gc));</pre><p>
        The generated column, <code class="literal">gc</code>, is defined as the
        expression <code class="literal">f1 + 1</code>. The column is also indexed
        and the optimizer can take that index into account during
        execution plan construction. In the following query, the
        <code class="literal">WHERE</code> clause refers to <code class="literal">gc</code>
        and the optimizer considers whether the index on that column
        yields a more efficient plan:
      </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 WHERE gc &gt; 9;</pre><p>
        The optimizer can use indexes on generated columns to generate
        execution plans, even in the absence of direct references in
        queries to those columns by name. This occurs if the
        <code class="literal">WHERE</code>, <code class="literal">ORDER BY</code>, or
        <code class="literal">GROUP BY</code> clause refers to an expression that
        matches the definition of some indexed generated column. The
        following query does not refer directly to <code class="literal">gc</code>
        but does use an expression that matches the definition of
        <code class="literal">gc</code>:
      </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 WHERE f1 + 1 &gt; 9;</pre><p>
        The optimizer recognizes that the expression <code class="literal">f1 +
        1</code> matches the definition of <code class="literal">gc</code> and
        that <code class="literal">gc</code> is indexed, so it considers that
        index during execution plan construction. You can see this using
        <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN SELECT * FROM t1 WHERE f1 + 1 &gt; 9\G</code></strong>
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: t1
   partitions: NULL
         type: range
possible_keys: gc
          key: gc
      key_len: 5
          ref: NULL
         rows: 1
     filtered: 100.00
        Extra: Using index condition
</pre><p>
        In effect, the optimizer has replaced the expression <code class="literal">f1
        + 1</code> with the name of the generated column that matches
        the expression. That is also apparent in the rewritten query
        available in the extended <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>
        information displayed by <a class="link" href="sql-statements.html#show-warnings" title="13.7.7.40 SHOW WARNINGS Statement"><code class="literal">SHOW
        WARNINGS</code></a>:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SHOW WARNINGS\G</code></strong>
*************************** 1. row ***************************
  Level: Note
   Code: 1003
Message: /* select#1 */ select `test`.`t1`.`f1` AS `f1`,`test`.`t1`.`gc`
         AS `gc` from `test`.`t1` where (`test`.`t1`.`gc` &gt; 9)
</pre><p>
        The following restrictions and conditions apply to the
        optimizer's use of generated column indexes:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            For a query expression to match a generated column
            definition, the expression must be identical and it must
            have the same result type. For example, if the generated
            column expression is <code class="literal">f1 + 1</code>, the
            optimizer will not recognize a match if the query uses
            <code class="literal">1 + f1</code>, or if <code class="literal">f1 + 1</code>
            (an integer expression) is compared with a string.
          </p></li><li class="listitem"><p>
            The optimization applies to these operators:
            <a class="link" href="functions.html#operator_equal"><code class="literal">=</code></a>,
            <a class="link" href="functions.html#operator_less-than"><code class="literal">&lt;</code></a>,
            <a class="link" href="functions.html#operator_less-than-or-equal"><code class="literal">&lt;=</code></a>,
            <a class="link" href="functions.html#operator_greater-than"><code class="literal">&gt;</code></a>,
            <a class="link" href="functions.html#operator_greater-than-or-equal"><code class="literal">&gt;=</code></a>,
            <a class="link" href="functions.html#operator_between"><code class="literal">BETWEEN</code></a>, and
            <a class="link" href="functions.html#operator_in"><code class="literal">IN()</code></a>.
          </p><p>
            For operators other than
            <a class="link" href="functions.html#operator_between"><code class="literal">BETWEEN</code></a> and
            <a class="link" href="functions.html#operator_in"><code class="literal">IN()</code></a>, either operand can be
            replaced by a matching generated column. For
            <a class="link" href="functions.html#operator_between"><code class="literal">BETWEEN</code></a> and
            <a class="link" href="functions.html#operator_in"><code class="literal">IN()</code></a>, only the first argument
            can be replaced by a matching generated column, and the
            other arguments must have the same result type.
            <a class="link" href="functions.html#operator_between"><code class="literal">BETWEEN</code></a> and
            <a class="link" href="functions.html#operator_in"><code class="literal">IN()</code></a> are not yet supported for
            comparisons involving JSON values.
          </p></li><li class="listitem"><p>
            The generated column must be defined as an expression that
            contains at least a function call or one of the operators
            mentioned in the preceding item. The expression cannot
            consist of a simple reference to another column. For
            example, <code class="literal">gc INT AS (f1) STORED</code> consists
            only of a column reference, so indexes on
            <code class="literal">gc</code> are not considered.
          </p></li><li class="listitem"><p>
            For comparisons of strings to indexed generated columns that
            compute a value from a JSON function that returns a quoted
            string, <a class="link" href="functions.html#function_json-unquote"><code class="literal">JSON_UNQUOTE()</code></a> is
            needed in the column definition to remove the extra quotes
            from the function value. (For direct comparison of a string
            to the function result, the JSON comparator handles quote
            removal, but this does not occur for index lookups.) For
            example, instead of writing a column definition like this:
          </p><pre data-lang="sql" class="programlisting">doc_name TEXT AS (JSON_EXTRACT(jdoc, '$.name')) STORED</pre><p>
            Write it like this:
          </p><pre data-lang="sql" class="programlisting">doc_name TEXT AS (JSON_UNQUOTE(JSON_EXTRACT(jdoc, '$.name'))) STORED</pre><p>
            With the latter definition, the optimizer can detect a match
            for both of these comparisons:
          </p><pre data-lang="sql" class="programlisting">... WHERE JSON_EXTRACT(jdoc, '$.name') = '<em class="replaceable"><code>some_string</code></em>' ...
... WHERE JSON_UNQUOTE(JSON_EXTRACT(jdoc, '$.name')) = '<em class="replaceable"><code>some_string</code></em>' ...
</pre><p>
            Without <a class="link" href="functions.html#function_json-unquote"><code class="literal">JSON_UNQUOTE()</code></a> in the
            column definition, the optimizer detects a match only for
            the first of those comparisons.
          </p></li><li class="listitem"><p>
            If the optimizer picks the wrong index, an index hint can be
            used to disable it and force the optimizer to make a
            different choice.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="invisible-indexes"></a>8.3.12 Invisible Indexes</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350432576"></a><p>
        MySQL supports invisible indexes; that is, indexes that are not
        used by the optimizer. The feature applies to indexes other than
        primary keys (either explicit or implicit).
      </p><p>
        Indexes are visible by default. To control index visibility
        explicitly for a new index, use a <code class="literal">VISIBLE</code> or
        <code class="literal">INVISIBLE</code> keyword as part of the index
        definition for <a class="link" href="sql-statements.html#create-table" title="13.1.20 CREATE TABLE Statement"><code class="literal">CREATE TABLE</code></a>,
        <a class="link" href="sql-statements.html#create-index" title="13.1.15 CREATE INDEX Statement"><code class="literal">CREATE INDEX</code></a>, or
        <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER TABLE</code></a>:
      </p><pre data-lang="sql" class="programlisting">CREATE TABLE t1 (
  i INT,
  j INT,
  k INT,
  INDEX i_idx (i) INVISIBLE
) ENGINE = InnoDB;
CREATE INDEX j_idx ON t1 (j) INVISIBLE;
ALTER TABLE t1 ADD INDEX k_idx (k) INVISIBLE;</pre><p>
        To alter the visibility of an existing index, use a
        <code class="literal">VISIBLE</code> or <code class="literal">INVISIBLE</code>
        keyword with the <code class="literal">ALTER TABLE ... ALTER INDEX</code>
        operation:
      </p><pre data-lang="sql" class="programlisting">ALTER TABLE t1 ALTER INDEX i_idx INVISIBLE;
ALTER TABLE t1 ALTER INDEX i_idx VISIBLE;</pre><p>
        Information about whether an index is visible or invisible is
        available from the
        <a class="link" href="information-schema.html#statistics-table" title="25.32 The INFORMATION_SCHEMA STATISTICS Table"><code class="literal">INFORMATION_SCHEMA.STATISTICS</code></a> table
        or <a class="link" href="sql-statements.html#show-index" title="13.7.7.22 SHOW INDEX Statement"><code class="literal">SHOW INDEX</code></a> output. For
        example:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT INDEX_NAME, IS_VISIBLE</code></strong>
       <strong class="userinput"><code>FROM INFORMATION_SCHEMA.STATISTICS</code></strong>
       <strong class="userinput"><code>WHERE TABLE_SCHEMA = 'db1' AND TABLE_NAME = 't1';</code></strong>
+------------+------------+
| INDEX_NAME | IS_VISIBLE |
+------------+------------+
| i_idx      | YES        |
| j_idx      | NO         |
| k_idx      | NO         |
+------------+------------+
</pre><p>
        Invisible indexes make it possible to test the effect of
        removing an index on query performance, without making a
        destructive change that must be undone should the index turn out
        to be required. Dropping and re-adding an index can be expensive
        for a large table, whereas making it invisible and visible are
        fast, in-place operations.
      </p><p>
        If an index made invisible actually is needed or used by the
        optimizer, there are several ways to notice the effect of its
        absence on queries for the table:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Errors occur for queries that include index hints that refer
            to the invisible index.
          </p></li><li class="listitem"><p>
            Performance Schema data shows an increase in workload for
            affected queries.
          </p></li><li class="listitem"><p>
            Queries have different
            <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> execution plans.
          </p></li><li class="listitem"><p>
            Queries appear in the slow query log that did not appear
            there previously.
</p></li></ul>
</div>
<a class="indexterm" name="idm46444350407648"></a><a class="indexterm" name="idm46444350406128"></a><p>
        The <code class="literal">use_invisible_indexes</code> flag of the
        <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
        variable controls whether the optimizer uses invisible indexes
        for query execution plan construction. If the flag is
        <code class="literal">off</code> (the default), the optimizer ignores
        invisible indexes (the same behavior as prior to the
        introduction of this flag). If the flag is
        <code class="literal">on</code>, invisible indexes remain invisible but
        the optimizer takes them into account for execution plan
        construction.
      </p><p>
        Index visibility does not affect index maintenance. For example,
        an index continues to be updated per changes to table rows, and
        a unique index prevents insertion of duplicates into a column,
        regardless of whether the index is visible or invisible.
      </p><p>
        A table with no explicit primary key may still have an effective
        implicit primary key if it has any <code class="literal">UNIQUE</code>
        indexes on <code class="literal">NOT NULL</code> columns. In this case,
        the first such index places the same constraint on table rows as
        an explicit primary key and that index cannot be made invisible.
        Consider the following table definition:
      </p><pre data-lang="sql" class="programlisting">CREATE TABLE t2 (
  i INT NOT NULL,
  j INT NOT NULL,
  UNIQUE j_idx (j)
) ENGINE = InnoDB;</pre><p>
        The definition includes no explicit primary key, but the index
        on <code class="literal">NOT NULL</code> column <code class="literal">j</code>
        places the same constraint on rows as a primary key and cannot
        be made invisible:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>ALTER TABLE t2 ALTER INDEX j_idx INVISIBLE;</code></strong>
ERROR 3522 (HY000): A primary key index cannot be invisible.
</pre><p>
        Now suppose that an explicit primary key is added to the table:
      </p><pre data-lang="sql" class="programlisting">ALTER TABLE t2 ADD PRIMARY KEY (i);</pre><p>
        The explicit primary key cannot be made invisible. In addition,
        the unique index on <code class="literal">j</code> no longer acts as an
        implicit primary key and as a result can be made invisible:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>ALTER TABLE t2 ALTER INDEX j_idx INVISIBLE;</code></strong>
Query OK, 0 rows affected (0.03 sec)
</pre>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="descending-indexes"></a>8.3.13 Descending Indexes</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350387296"></a><a class="indexterm" name="idm46444350386224"></a><p>
        MySQL supports descending indexes: <code class="literal">DESC</code> in an
        index definition is no longer ignored but causes storage of key
        values in descending order. Previously, indexes could be scanned
        in reverse order but at a performance penalty. A descending
        index can be scanned in forward order, which is more efficient.
        Descending indexes also make it possible for the optimizer to
        use multiple-column indexes when the most efficient scan order
        mixes ascending order for some columns and descending order for
        others.
      </p><p>
        Consider the following table definition, which contains two
        columns and four two-column index definitions for the various
        combinations of ascending and descending indexes on the columns:
      </p><pre data-lang="sql" class="programlisting">CREATE TABLE t (
  c1 INT, c2 INT,
  INDEX idx1 (c1 ASC, c2 ASC),
  INDEX idx2 (c1 ASC, c2 DESC),
  INDEX idx3 (c1 DESC, c2 ASC),
  INDEX idx4 (c1 DESC, c2 DESC)
);</pre><p>
        The table definition results in four distinct indexes. The
        optimizer can perform a forward index scan for each of the
        <code class="literal">ORDER BY</code> clauses and need not use a
        <code class="literal">filesort</code> operation:
      </p><pre data-lang="sql" class="programlisting">ORDER BY c1 ASC, c2 ASC    -- optimizer can use idx1
ORDER BY c1 DESC, c2 DESC  -- optimizer can use idx4
ORDER BY c1 ASC, c2 DESC   -- optimizer can use idx2
ORDER BY c1 DESC, c2 ASC   -- optimizer can use idx3</pre><p>
        Use of descending indexes is subject to these conditions:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Descending indexes are supported only for the
            <code class="literal">InnoDB</code> storage engine, with these
            limitations:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                Change buffering is not supported for a secondary index
                if the index contains a descending index key column or
                if the primary key includes a descending index column.
              </p></li><li class="listitem"><p>
                The <code class="literal">InnoDB</code> SQL parser does not use
                descending indexes. For <code class="literal">InnoDB</code>
                full-text search, this means that the index required on
                the <code class="literal">FTS_DOC_ID</code> column of the indexed
                table cannot be defined as a descending index. For more
                information, see
                <a class="xref" href="innodb-storage-engine.html#innodb-fulltext-index" title="15.6.2.4 InnoDB FULLTEXT Indexes">Section 15.6.2.4, “InnoDB FULLTEXT Indexes”</a>.
</p></li></ul>
</div>
</li><li class="listitem"><p>
            Descending indexes are supported for all data types for
            which ascending indexes are available.
          </p></li><li class="listitem"><p>
            Descending indexes are supported for ordinary (nongenerated)
            and generated columns (both <code class="literal">VIRTUAL</code> and
            <code class="literal">STORED</code>).
          </p></li><li class="listitem"><p>
            <code class="literal">DISTINCT</code> can use any index containing
            matching columns, including descending key parts.
          </p></li><li class="listitem"><p>
            Indexes that have descending key parts are not used for
            <a class="link" href="functions.html#function_min"><code class="literal">MIN()</code></a>/<a class="link" href="functions.html#function_max"><code class="literal">MAX()</code></a>
            optimization of queries that invoke aggregate functions but
            do not have a <code class="literal">GROUP BY</code> clause.
          </p></li><li class="listitem"><p>
            Descending indexes are supported for
            <code class="literal">BTREE</code> but not <code class="literal">HASH</code>
            indexes. Descending indexes are not supported for
            <code class="literal">FULLTEXT</code> or <code class="literal">SPATIAL</code>
            indexes.
          </p><p>
            Explicitly specified <code class="literal">ASC</code> and
            <code class="literal">DESC</code> designators for
            <code class="literal">HASH</code>, <code class="literal">FULLTEXT</code>, and
            <code class="literal">SPATIAL</code> indexes results in an error.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="timestamp-lookups"></a>8.3.14 Indexed Lookups from TIMESTAMP Columns</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350351888"></a><a class="indexterm" name="idm46444350350432"></a><p>
        Temporal values are stored in
        <a class="link" href="data-types.html#datetime" title="11.2.2 The DATE, DATETIME, and TIMESTAMP Types"><code class="literal">TIMESTAMP</code></a> columns as UTC values,
        and values inserted into and retrieved from
        <a class="link" href="data-types.html#datetime" title="11.2.2 The DATE, DATETIME, and TIMESTAMP Types"><code class="literal">TIMESTAMP</code></a> columns are converted
        between the session time zone and UTC. (This is the same type of
        conversion performed by the
        <a class="link" href="functions.html#function_convert-tz"><code class="literal">CONVERT_TZ()</code></a> function. If the
        session time zone is UTC, there is effectively no time zone
        conversion.)
      </p><a class="indexterm" name="idm46444350344512"></a><p>
        Due to conventions for local time zone changes such as Daylight
        Saving Time (DST), conversions between UTC and non-UTC time
        zones are not one-to-one in both directions. UTC values that are
        distinct may not be distinct in another time zone. The following
        example shows distinct UTC values that become identical in a
        non-UTC time zone:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>CREATE TABLE tstable (ts TIMESTAMP);</code></strong>
mysql&gt; <strong class="userinput"><code>SET time_zone = 'UTC'; -- insert UTC values</code></strong>
mysql&gt; <strong class="userinput"><code>INSERT INTO tstable VALUES</code></strong>
       <strong class="userinput"><code>('2018-10-28 00:30:00'),</code></strong>
       <strong class="userinput"><code>('2018-10-28 01:30:00');</code></strong>
mysql&gt; <strong class="userinput"><code>SELECT ts FROM tstable;</code></strong>
+---------------------+
| ts                  |
+---------------------+
| 2018-10-28 00:30:00 |
| 2018-10-28 01:30:00 |
+---------------------+
mysql&gt; <strong class="userinput"><code>SET time_zone = 'MET'; -- retrieve non-UTC values</code></strong>
mysql&gt; <strong class="userinput"><code>SELECT ts FROM tstable;</code></strong>
+---------------------+
| ts                  |
+---------------------+
| 2018-10-28 02:30:00 |
| 2018-10-28 02:30:00 |
+---------------------+
</pre>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
          To use named time zones such as <code class="literal">'MET'</code> or
          <code class="literal">'Europe/Amsterdam'</code>, the time zone tables
          must be properly set up. For instructions, see
          <a class="xref" href="server-administration.html#time-zone-support" title="5.1.14 MySQL Server Time Zone Support">Section 5.1.14, “MySQL Server Time Zone Support”</a>.
</p>
</div>
<p>
        You can see that the two distinct UTC values are the same when
        converted to the <code class="literal">'MET'</code> time zone. This
        phenomenon can lead to different results for a given
        <a class="link" href="data-types.html#datetime" title="11.2.2 The DATE, DATETIME, and TIMESTAMP Types"><code class="literal">TIMESTAMP</code></a> column query, depending
        on whether the optimizer uses an index to execute the query.
      </p><p>
        Suppose that a query selects values from the table shown earlier
        using a <code class="literal">WHERE</code> clause to search the
        <code class="literal">ts</code> column for a single specific value such as
        a user-provided timestamp literal:
      </p><pre data-lang="sql" class="programlisting">SELECT ts FROM tstable
WHERE ts = '<em class="replaceable"><code>literal</code></em>';
</pre><p>
        Suppose further that the query executes under these conditions:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            The session time zone is not UTC and has a DST shift. For
            example:
          </p><pre data-lang="sql" class="programlisting">SET time_zone = 'MET';</pre></li><li class="listitem"><p>
            Unique UTC values stored in the
            <a class="link" href="data-types.html#datetime" title="11.2.2 The DATE, DATETIME, and TIMESTAMP Types"><code class="literal">TIMESTAMP</code></a> column are not
            unique in the session time zone due to DST shifts. (The
            example shown earlier illustrates how this can occur.)
          </p></li><li class="listitem"><p>
            The query specifies a search value that is within the hour
            of entry into DST in the session time zone.
</p></li></ul>
</div>
<p>
        Under those conditions, the comparison in the
        <code class="literal">WHERE</code> clause occurs in different ways for
        nonindexed and indexed lookups and leads to different results:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            If there is no index or the optimizer cannot use it,
            comparisons occur in the session time zone. The optimizer
            performs a table scan in which it retrieves each
            <code class="literal">ts</code> column value, converts it from UTC to
            the session time zone, and compares it to the search value
            (also interpreted in the session time zone):
          </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT ts FROM tstable</code></strong>
       <strong class="userinput"><code>WHERE ts = '2018-10-28 02:30:00';</code></strong>
+---------------------+
| ts                  |
+---------------------+
| 2018-10-28 02:30:00 |
| 2018-10-28 02:30:00 |
+---------------------+
</pre><p>
            Because the stored <code class="literal">ts</code> values are
            converted to the session time zone, it is possible for the
            query to return two timestamp values that are distinct as
            UTC values but equal in the session time zone: One value
            that occurs before the DST shift when clocks are changed,
            and one value that was occurs after the DST shift.
          </p></li><li class="listitem"><p>
            If there is a usable index, comparisons occur in UTC. The
            optimizer performs an index scan, first converting the
            search value from the session time zone to UTC, then
            comparing the result to the UTC index entries:
          </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>ALTER TABLE tstable ADD INDEX (ts);</code></strong>
mysql&gt; <strong class="userinput"><code>SELECT ts FROM tstable</code></strong>
       <strong class="userinput"><code>WHERE ts = '2018-10-28 02:30:00';</code></strong>
+---------------------+
| ts                  |
+---------------------+
| 2018-10-28 02:30:00 |
+---------------------+
</pre><p>
            In this case, the (converted) search value is matched only
            to index entries, and because the index entries for the
            distinct stored UTC values are also distinct, the search
            value can match only one of them.
</p></li></ul>
</div>
<p>
        Due to different optimizer operation for nonindexed and indexed
        lookups, the query produces different results in each case. The
        result from the nonindexed lookup returns all values that match
        in the session time zone. The indexed lookup cannot do so:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            It is performed within the storage engine, which knows only
            about UTC values.
          </p></li><li class="listitem"><p>
            For the two distinct session time zone values that map to
            the same UTC value, the indexed lookup matches only the
            corresponding UTC index entry and returns only a single row.
</p></li></ul>
</div>
<p>
        In the preceding discussion, the data set stored in
        <code class="literal">tstable</code> happens to consist of distinct UTC
        values. In such cases, all index-using queries of the form shown
        match at most one index entry.
      </p><p>
        If the index is not <code class="literal">UNIQUE</code>, it is possible
        for the table (and the index) to store multiple instances of a
        given UTC value. For example, the <code class="literal">ts</code> column
        might contain multiple instances of the UTC value
        <code class="literal">'2018-10-28 00:30:00'</code>. In this case, the
        index-using query would return each of them (converted to the
        MET value <code class="literal">'2018-10-28 02:30:00'</code> in the result
        set). It remains true that index-using queries match the
        converted search value to a single value in the UTC index
        entries, rather than matching multiple UTC values that convert
        to the search value in the session time zone.
      </p><p>
        If it is important to return all <code class="literal">ts</code> values
        that match in the session time zone, the workaround is to
        suppress use of the index with an <code class="literal">IGNORE
        INDEX</code> hint:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT ts FROM tstable</code></strong>
       <strong class="userinput"><code>IGNORE INDEX (ts)</code></strong>
       <strong class="userinput"><code>WHERE ts = '2018-10-28 02:30:00';</code></strong>
+---------------------+
| ts                  |
+---------------------+
| 2018-10-28 02:30:00 |
| 2018-10-28 02:30:00 |
+---------------------+
</pre><p>
        The same lack of one-to-one mapping for time zone conversions in
        both directions occurs in other contexts as well, such as
        conversions performed with the
        <a class="link" href="functions.html#function_from-unixtime"><code class="literal">FROM_UNIXTIME()</code></a> and
        <a class="link" href="functions.html#function_unix-timestamp"><code class="literal">UNIX_TIMESTAMP()</code></a> functions. See
        <a class="xref" href="functions.html#date-and-time-functions" title="12.6 Date and Time Functions">Section 12.6, “Date and Time Functions”</a>.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a name="optimizing-database-structure"></a>8.4 Optimizing Database Structure</h2>

</div>

</div>

</div>
<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#data-size">8.4.1 Optimizing Data Size</a></span></dt><dt><span class="section"><a href="optimization.html#optimize-data-types">8.4.2 Optimizing MySQL Data Types</a></span></dt><dt><span class="section"><a href="optimization.html#optimize-multi-tables">8.4.3 Optimizing for Many Tables</a></span></dt><dt><span class="section"><a href="optimization.html#internal-temporary-tables">8.4.4 Internal Temporary Table Use in MySQL</a></span></dt><dt><span class="section"><a href="optimization.html#database-count-limit">8.4.5 Limits on Number of Databases and Tables</a></span></dt><dt><span class="section"><a href="optimization.html#table-size-limit">8.4.6 Limits on Table Size</a></span></dt><dt><span class="section"><a href="optimization.html#column-count-limit">8.4.7 Limits on Table Column Count and Row Size</a></span></dt></dl>
</div>
<p>
      In your role as a database designer, look for the most efficient
      way to organize your schemas, tables, and columns. As when tuning
      application code, you minimize I/O, keep related items together,
      and plan ahead so that performance stays high as the data volume
      increases. Starting with an efficient database design makes it
      easier for team members to write high-performing application code,
      and makes the database likely to endure as applications evolve and
      are rewritten.
</p>
<div class="section">

<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="data-size"></a>8.4.1 Optimizing Data Size</h3>
</div>
</div>
</div>
<a class="indexterm" name="idm46444350287616"></a><a class="indexterm" name="idm46444350286128"></a><a class="indexterm" name="idm46444350284640"></a><a class="indexterm" name="idm46444350283152"></a><a class="indexterm" name="idm46444350281664"></a><p>
        Design your tables to minimize their space on the disk. This can
        result in huge improvements by reducing the amount of data
        written to and read from disk. Smaller tables normally require
        less main memory while their contents are being actively
        processed during query execution. Any space reduction for table
        data also results in smaller indexes that can be processed
        faster.
      </p><p>
        MySQL supports many different storage engines (table types) and
        row formats. For each table, you can decide which storage and
        indexing method to use. Choosing the proper table format for
        your application can give you a big performance gain. See
        <a class="xref" href="innodb-storage-engine.html" title="Chapter 15 The InnoDB Storage Engine">Chapter 15, <i>The InnoDB Storage Engine</i></a>, and
        <a class="xref" href="storage-engines.html" title="Chapter 16 Alternative Storage Engines">Chapter 16, <i>Alternative Storage Engines</i></a>.
      </p><p>
        You can get better performance for a table and minimize storage
        space by using the techniques listed here:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a class="xref" href="optimization.html#data-size-table-columns" title="Table Columns">Table Columns</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#data-size-row-format" title="Row Format">Row Format</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#data-size-indexes" title="Indexes">Indexes</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#data-size-joins" title="Joins">Joins</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#data-size-normalization" title="Normalization">Normalization</a></p></li></ul>
</div>

<div class="simplesect">

<div class="titlepage">
<div>

<div class="simple">
<h4 class="title"><a name="data-size-table-columns"></a>Table Columns</h4>

</div>

</div>

</div>

<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Use the most efficient (smallest) data types possible.
              MySQL has many specialized types that save disk space and
              memory. For example, use the smaller integer types if
              possible to get smaller tables.
              <a class="link" href="data-types.html#integer-types" title="11.1.2 Integer Types (Exact Value) - INTEGER, INT, SMALLINT, TINYINT, MEDIUMINT, BIGINT"><code class="literal">MEDIUMINT</code></a> is often a better
              choice than <a class="link" href="data-types.html#integer-types" title="11.1.2 Integer Types (Exact Value) - INTEGER, INT, SMALLINT, TINYINT, MEDIUMINT, BIGINT"><code class="literal">INT</code></a> because a
              <a class="link" href="data-types.html#integer-types" title="11.1.2 Integer Types (Exact Value) - INTEGER, INT, SMALLINT, TINYINT, MEDIUMINT, BIGINT"><code class="literal">MEDIUMINT</code></a> column uses 25%
              less space.
            </p></li><li class="listitem"><p>
              Declare columns to be <code class="literal">NOT NULL</code> if
              possible. It makes SQL operations faster, by enabling
              better use of indexes and eliminating overhead for testing
              whether each value is <code class="literal">NULL</code>. You also
              save some storage space, one bit per column. If you really
              need <code class="literal">NULL</code> values in your tables, use
              them. Just avoid the default setting that allows
              <code class="literal">NULL</code> values in every column.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="data-size-row-format"></a>Row Format</h4>

</div>

</div>

</div>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <code class="literal">InnoDB</code> tables are created using the
              <code class="literal">DYNAMIC</code> row format by default. To use a
              row format other than <code class="literal">DYNAMIC</code>,
              configure
              <a class="link" href="innodb-storage-engine.html#sysvar_innodb_default_row_format"><code class="literal">innodb_default_row_format</code></a>,
              or specify the <code class="literal">ROW_FORMAT</code> option
              explicitly in a <a class="link" href="sql-statements.html#create-table" title="13.1.20 CREATE TABLE Statement"><code class="literal">CREATE
              TABLE</code></a> or <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER
              TABLE</code></a> statement.
            </p><p>
              The compact family of row formats, which includes
              <code class="literal">COMPACT</code>, <code class="literal">DYNAMIC</code>,
              and <code class="literal">COMPRESSED</code>, decreases row storage
              space at the cost of increasing CPU use for some
              operations. If your workload is a typical one that is
              limited by cache hit rates and disk speed it is likely to
              be faster. If it is a rare case that is limited by CPU
              speed, it might be slower.
            </p><p>
              The compact family of row formats also optimizes
              <a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">CHAR</code></a> column storage when
              using a variable-length character set such as
              <code class="literal">utf8mb3</code> or <code class="literal">utf8mb4</code>.
              With <code class="literal">ROW_FORMAT=REDUNDANT</code>,
              <code class="literal">CHAR(<em class="replaceable"><code>N</code></em>)</code>
              occupies <em class="replaceable"><code>N</code></em> × the maximum
              byte length of the character set. Many languages can be
              written primarily using single-byte
              <code class="literal">utf8</code> characters, so a fixed storage
              length often wastes space. With the compact family of rows
              formats, <code class="literal">InnoDB</code> allocates a variable
              amount of storage in the range of
              <em class="replaceable"><code>N</code></em> to
              <em class="replaceable"><code>N</code></em> × the maximum byte
              length of the character set for these columns by stripping
              trailing spaces. The minimum storage length is
              <em class="replaceable"><code>N</code></em> bytes to facilitate in-place
              updates in typical cases. For more information, see
              <a class="xref" href="innodb-storage-engine.html#innodb-row-format" title="15.10 InnoDB Row Formats">Section 15.10, “InnoDB Row Formats”</a>.
            </p></li><li class="listitem"><p>
              To minimize space even further by storing table data in
              compressed form, specify
              <code class="literal">ROW_FORMAT=COMPRESSED</code> when creating
              <code class="literal">InnoDB</code> tables, or run the
              <a class="link" href="programs.html#myisampack" title="4.6.6 myisampack — Generate Compressed, Read-Only MyISAM Tables"><span class="command"><strong>myisampack</strong></span></a> command on an existing
              <code class="literal">MyISAM</code> table.
              (<code class="literal">InnoDB</code> compressed tables are readable
              and writable, while <code class="literal">MyISAM</code> compressed
              tables are read-only.)
            </p></li><li class="listitem"><p>
              For <code class="literal">MyISAM</code> tables, if you do not have
              any variable-length columns
              (<a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">VARCHAR</code></a>,
              <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">TEXT</code></a>, or
              <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a> columns), a fixed-size
              row format is used. This is faster but may waste some
              space. See <a class="xref" href="storage-engines.html#myisam-table-formats" title="16.2.3 MyISAM Table Storage Formats">Section 16.2.3, “MyISAM Table Storage Formats”</a>. You can
              hint that you want to have fixed length rows even if you
              have <a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">VARCHAR</code></a> columns with
              the <a class="link" href="sql-statements.html#create-table" title="13.1.20 CREATE TABLE Statement"><code class="literal">CREATE TABLE</code></a> option
              <code class="literal">ROW_FORMAT=FIXED</code>.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="data-size-indexes"></a>Indexes</h4>

</div>

</div>

</div>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              The primary index of a table should be as short as
              possible. This makes identification of each row easy and
              efficient. For <code class="literal">InnoDB</code> tables, the
              primary key columns are duplicated in each secondary index
              entry, so a short primary key saves considerable space if
              you have many secondary indexes.
            </p></li><li class="listitem"><p>
              Create only the indexes that you need to improve query
              performance. Indexes are good for retrieval, but slow down
              insert and update operations. If you access a table mostly
              by searching on a combination of columns, create a single
              composite index on them rather than a separate index for
              each column. The first part of the index should be the
              column most used. If you <span class="emphasis"><em>always</em></span> use
              many columns when selecting from the table, the first
              column in the index should be the one with the most
              duplicates, to obtain better compression of the index.
            </p></li><li class="listitem"><p>
              If it is very likely that a long string column has a
              unique prefix on the first number of characters, it is
              better to index only this prefix, using MySQL's
              support for creating an index on the leftmost part of the
              column (see <a class="xref" href="sql-statements.html#create-index" title="13.1.15 CREATE INDEX Statement">Section 13.1.15, “CREATE INDEX Statement”</a>). Shorter
              indexes are faster, not only because they require less
              disk space, but because they also give you more hits in
              the index cache, and thus fewer disk seeks. See
              <a class="xref" href="server-administration.html#server-configuration" title="5.1.1 Configuring the Server">Section 5.1.1, “Configuring the Server”</a>.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="data-size-joins"></a>Joins</h4>

</div>

</div>

</div>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              In some circumstances, it can be beneficial to split into
              two a table that is scanned very often. This is especially
              true if it is a dynamic-format table and it is possible to
              use a smaller static format table that can be used to find
              the relevant rows when scanning the table.
            </p></li><li class="listitem"><p>
              Declare columns with identical information in different
              tables with identical data types, to speed up joins based
              on the corresponding columns.
            </p></li><li class="listitem"><p>
              Keep column names simple, so that you can use the same
              name across different tables and simplify join queries.
              For example, in a table named <code class="literal">customer</code>,
              use a column name of <code class="literal">name</code> instead of
              <code class="literal">customer_name</code>. To make your names
              portable to other SQL servers, consider keeping them
              shorter than 18 characters.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="data-size-normalization"></a>Normalization</h4>

</div>

</div>

</div>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Normally, try to keep all data nonredundant (observing
              what is referred to in database theory as
              <span class="firstterm">third normal form</span>).
              Instead of repeating lengthy values such as names and
              addresses, assign them unique IDs, repeat these IDs as
              needed across multiple smaller tables, and join the tables
              in queries by referencing the IDs in the join clause.
            </p></li><li class="listitem"><p>
              If speed is more important than disk space and the
              maintenance costs of keeping multiple copies of data, for
              example in a business intelligence scenario where you
              analyze all the data from large tables, you can relax the
              normalization rules, duplicating information or creating
              summary tables to gain more speed.
</p></li></ul>
</div>

</div>

</div>

<div class="section">

<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="optimize-data-types"></a>8.4.2 Optimizing MySQL Data Types</h3>

</div>

</div>

</div>

<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#optimize-numeric">8.4.2.1 Optimizing for Numeric Data</a></span></dt><dt><span class="section"><a href="optimization.html#optimize-character">8.4.2.2 Optimizing for Character and String Types</a></span></dt><dt><span class="section"><a href="optimization.html#optimize-blob">8.4.2.3 Optimizing for BLOB Types</a></span></dt></dl>
</div>

<div class="section">

<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="optimize-numeric"></a>8.4.2.1 Optimizing for Numeric Data</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350200960"></a>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              For unique IDs or other values that can be represented as
              either strings or numbers, prefer numeric columns to
              string columns. Since large numeric values can be stored
              in fewer bytes than the corresponding strings, it is
              faster and takes less memory to transfer and compare them.
            </p></li><li class="listitem"><p>
              If you are using numeric data, it is faster in many cases
              to access information from a database (using a live
              connection) than to access a text file. Information in the
              database is likely to be stored in a more compact format
              than in the text file, so accessing it involves fewer disk
              accesses. You also save code in your application because
              you can avoid parsing the text file to find line and
              column boundaries.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="optimize-character"></a>8.4.2.2 Optimizing for Character and String Types</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350195344"></a><p>
          For character and string columns, follow these guidelines:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Use binary collation order for fast comparison and sort
              operations, when you do not need language-specific
              collation features. You can use the
              <a class="link" href="functions.html#operator_binary"><code class="literal">BINARY</code></a> operator to use binary
              collation within a particular query.
            </p></li><li class="listitem"><p>
              When comparing values from different columns, declare
              those columns with the same character set and collation
              wherever possible, to avoid string conversions while
              running the query.
            </p></li><li class="listitem"><p>
              For column values less than 8KB in size, use binary
              <code class="literal">VARCHAR</code> instead of
              <code class="literal">BLOB</code>. The <code class="literal">GROUP BY</code>
              and <code class="literal">ORDER BY</code> clauses can generate
              temporary tables, and these temporary tables can use the
              <code class="literal">MEMORY</code> storage engine if the original
              table does not contain any <code class="literal">BLOB</code>
              columns.
            </p></li><li class="listitem"><p>
              If a table contains string columns such as name and
              address, but many queries do not retrieve those columns,
              consider splitting the string columns into a separate
              table and using join queries with a foreign key when
              necessary. When MySQL retrieves any value from a row, it
              reads a data block containing all the columns of that row
              (and possibly other adjacent rows). Keeping each row
              small, with only the most frequently used columns, allows
              more rows to fit in each data block. Such compact tables
              reduce disk I/O and memory usage for common queries.
            </p></li><li class="listitem"><p>
              When you use a randomly generated value as a primary key
              in an <code class="literal">InnoDB</code> table, prefix it with an
              ascending value such as the current date and time if
              possible. When consecutive primary values are physically
              stored near each other, <code class="literal">InnoDB</code> can
              insert and retrieve them faster.
            </p></li><li class="listitem"><p>
              See <a class="xref" href="optimization.html#optimize-numeric" title="8.4.2.1 Optimizing for Numeric Data">Section 8.4.2.1, “Optimizing for Numeric Data”</a> for reasons why a
              numeric column is usually preferable to an equivalent
              string column.
</p></li></ul>
</div>

</div>

<div class="section">

<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="optimize-blob"></a>8.4.2.3 Optimizing for BLOB Types</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350178000"></a>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              When storing a large blob containing textual data,
              consider compressing it first. Do not use this technique
              when the entire table is compressed by
              <code class="literal">InnoDB</code> or <code class="literal">MyISAM</code>.
            </p></li><li class="listitem"><p>
              For a table with several columns, to reduce memory
              requirements for queries that do not use the BLOB column,
              consider splitting the BLOB column into a separate table
              and referencing it with a join query when needed.
            </p></li><li class="listitem"><p>
              Since the performance requirements to retrieve and display
              a BLOB value might be very different from other data
              types, you could put the BLOB-specific table on a
              different storage device or even a separate database
              instance. For example, to retrieve a BLOB might require a
              large sequential disk read that is better suited to a
              traditional hard drive than to an
              <a class="link" href="glossary.html#glos_ssd" title="SSD">SSD device</a>.
            </p></li><li class="listitem"><p>
              See <a class="xref" href="optimization.html#optimize-character" title="8.4.2.2 Optimizing for Character and String Types">Section 8.4.2.2, “Optimizing for Character and String Types”</a> for reasons why a
              binary <code class="literal">VARCHAR</code> column is sometimes
              preferable to an equivalent BLOB column.
            </p></li><li class="listitem"><p>
              Rather than testing for equality against a very long text
              string, you can store a hash of the column value in a
              separate column, index that column, and test the hashed
              value in queries. (Use the <code class="literal">MD5()</code> or
              <code class="literal">CRC32()</code> function to produce the hash
              value.) Since hash functions can produce duplicate results
              for different inputs, you still include a clause
              <code class="literal">AND <em class="replaceable"><code>blob_column</code></em> =
              <em class="replaceable"><code>long_string_value</code></em></code> in
              the query to guard against false matches; the performance
              benefit comes from the smaller, easily scanned index for
              the hashed values.
</p></li></ul>
</div>

</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="optimize-multi-tables"></a>8.4.3 Optimizing for Many Tables</h3>

</div>

</div>

</div>
<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#table-cache">8.4.3.1 How MySQL Opens and Closes Tables</a></span></dt><dt><span class="section"><a href="optimization.html#creating-many-tables">8.4.3.2 Disadvantages of Creating Many Tables in the Same Database</a></span></dt></dl>
</div>
<a class="indexterm" name="idm46444350162224"></a><p>
        Some techniques for keeping individual queries fast involve
        splitting data across many tables. When the number of tables
        runs into the thousands or even millions, the overhead of
        dealing with all these tables becomes a new performance
        consideration.
</p>
<div class="section">

<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="table-cache"></a>8.4.3.1 How MySQL Opens and Closes Tables</h4>
</div>
</div>
</div>
<a class="indexterm" name="idm46444350158912"></a><a class="indexterm" name="idm46444350157872"></a><a class="indexterm" name="idm46444350156384"></a><a class="indexterm" name="idm46444350154896"></a><a class="indexterm" name="idm46444350153408"></a><a class="indexterm" name="idm46444350151920"></a><a class="indexterm" name="idm46444350150848"></a><a class="indexterm" name="idm46444350149360"></a><p>
          When you execute a <a class="link" href="programs.html#mysqladmin" title="4.5.2 mysqladmin — Client for Administering a MySQL Server"><span class="command"><strong>mysqladmin status</strong></span></a>
          command, you should see something like this:
        </p><pre data-lang="none" class="programlisting">Uptime: 426 Running threads: 1 Questions: 11082
Reloads: 1 Open tables: 12</pre><p>
          The <code class="literal">Open tables</code> value of 12 can be somewhat
          puzzling if you have fewer than 12 tables.
        </p><p>
          MySQL is multithreaded, so there may be many clients issuing
          queries for a given table simultaneously. To minimize the
          problem with multiple client sessions having different states
          on the same table, the table is opened independently by each
          concurrent session. This uses additional memory but normally
          increases performance. With <code class="literal">MyISAM</code> tables,
          one extra file descriptor is required for the data file for
          each client that has the table open. (By contrast, the index
          file descriptor is shared between all sessions.)
        </p><p>
          The <a class="link" href="server-administration.html#sysvar_table_open_cache"><code class="literal">table_open_cache</code></a> and
          <a class="link" href="server-administration.html#sysvar_max_connections"><code class="literal">max_connections</code></a> system
          variables affect the maximum number of files the server keeps
          open. If you increase one or both of these values, you may run
          up against a limit imposed by your operating system on the
          per-process number of open file descriptors. Many operating
          systems permit you to increase the open-files limit, although
          the method varies widely from system to system. Consult your
          operating system documentation to determine whether it is
          possible to increase the limit and how to do so.
        </p><p>
          <a class="link" href="server-administration.html#sysvar_table_open_cache"><code class="literal">table_open_cache</code></a> is related
          to <a class="link" href="server-administration.html#sysvar_max_connections"><code class="literal">max_connections</code></a>. For
          example, for 200 concurrent running connections, specify a
          table cache size of at least <code class="literal">200 *
          <em class="replaceable"><code>N</code></em></code>, where
          <em class="replaceable"><code>N</code></em> is the maximum number of tables
          per join in any of the queries which you execute. You must
          also reserve some extra file descriptors for temporary tables
          and files.
        </p><p>
          Make sure that your operating system can handle the number of
          open file descriptors implied by the
          <a class="link" href="server-administration.html#sysvar_table_open_cache"><code class="literal">table_open_cache</code></a> setting. If
          <a class="link" href="server-administration.html#sysvar_table_open_cache"><code class="literal">table_open_cache</code></a> is set too
          high, MySQL may run out of file descriptors and exhibit
          symptoms such as refusing connections or failing to perform
          queries.
        </p><p>
          Also take into account that the <code class="literal">MyISAM</code>
          storage engine needs two file descriptors for each unique open
          table. To increase the number of file descriptors available to
          MySQL, set the
          <a class="link" href="server-administration.html#sysvar_open_files_limit"><code class="literal">open_files_limit</code></a> system
          variable. See <a class="xref" href="error-handling.html#not-enough-file-handles" title="B.4.2.17 File Not Found and Similar Errors">Section B.4.2.17, “File Not Found and Similar Errors”</a>.
        </p><p>
          The cache of open tables is kept at a level of
          <a class="link" href="server-administration.html#sysvar_table_open_cache"><code class="literal">table_open_cache</code></a> entries. The
          server autosizes the cache size at startup. To set the size
          explicitly, set the
          <a class="link" href="server-administration.html#sysvar_table_open_cache"><code class="literal">table_open_cache</code></a> system
          variable at startup. MySQL may temporarily open more tables
          than this to execute queries, as described later in this
          section.
        </p><p>
          MySQL closes an unused table and removes it from the table
          cache under the following circumstances:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              When the cache is full and a thread tries to open a table
              that is not in the cache.
            </p></li><li class="listitem"><p>
              When the cache contains more than
              <a class="link" href="server-administration.html#sysvar_table_open_cache"><code class="literal">table_open_cache</code></a> entries
              and a table in the cache is no longer being used by any
              threads.
            </p></li><li class="listitem"><p>
              When a table-flushing operation occurs. This happens when
              someone issues a <a class="link" href="sql-statements.html#flush-tables"><code class="literal">FLUSH
              TABLES</code></a> statement or executes a
              <a class="link" href="programs.html#mysqladmin" title="4.5.2 mysqladmin — Client for Administering a MySQL Server"><span class="command"><strong>mysqladmin flush-tables</strong></span></a> or
              <a class="link" href="programs.html#mysqladmin" title="4.5.2 mysqladmin — Client for Administering a MySQL Server"><span class="command"><strong>mysqladmin refresh</strong></span></a> command.
</p></li></ul>
</div>
<p>
          When the table cache fills up, the server uses the following
          procedure to locate a cache entry to use:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Tables not currently in use are released, beginning with
              the table least recently used.
            </p></li><li class="listitem"><p>
              If a new table must be opened, but the cache is full and
              no tables can be released, the cache is temporarily
              extended as necessary. When the cache is in a temporarily
              extended state and a table goes from a used to unused
              state, the table is closed and released from the cache.
</p></li></ul>
</div>
<p>
          A <code class="literal">MyISAM</code> table is opened for each
          concurrent access. This means the table needs to be opened
          twice if two threads access the same table or if a thread
          accesses the table twice in the same query (for example, by
          joining the table to itself). Each concurrent open requires an
          entry in the table cache. The first open of any
          <code class="literal">MyISAM</code> table takes two file descriptors:
          one for the data file and one for the index file. Each
          additional use of the table takes only one file descriptor for
          the data file. The index file descriptor is shared among all
          threads.
        </p><p>
          If you are opening a table with the <code class="literal">HANDLER
          <em class="replaceable"><code>tbl_name</code></em> OPEN</code> statement,
          a dedicated table object is allocated for the thread. This
          table object is not shared by other threads and is not closed
          until the thread calls <code class="literal">HANDLER
          <em class="replaceable"><code>tbl_name</code></em> CLOSE</code> or the
          thread terminates. When this happens, the table is put back in
          the table cache (if the cache is not full). See
          <a class="xref" href="sql-statements.html#handler" title="13.2.4 HANDLER Statement">Section 13.2.4, “HANDLER Statement”</a>.
        </p><p>
          To determine whether your table cache is too small, check the
          <a class="link" href="server-administration.html#statvar_Opened_tables"><code class="literal">Opened_tables</code></a> status
          variable, which indicates the number of table-opening
          operations since the server started:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SHOW GLOBAL STATUS LIKE 'Opened_tables';</code></strong>
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Opened_tables | 2741  |
+---------------+-------+
</pre><p>
          If the value is very large or increases rapidly, even when you
          have not issued many <a class="link" href="sql-statements.html#flush-tables"><code class="literal">FLUSH
          TABLES</code></a> statements, increase the
          <a class="link" href="server-administration.html#sysvar_table_open_cache"><code class="literal">table_open_cache</code></a> value at
          server startup.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="creating-many-tables"></a>8.4.3.2 Disadvantages of Creating Many Tables in the Same Database</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350098896"></a><p>
          If you have many <code class="literal">MyISAM</code> tables in the same
          database directory, open, close, and create operations are
          slow. If you execute <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>
          statements on many different tables, there is a little
          overhead when the table cache is full, because for every table
          that has to be opened, another must be closed. You can reduce
          this overhead by increasing the number of entries permitted in
          the table cache.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="internal-temporary-tables"></a>8.4.4 Internal Temporary Table Use in MySQL</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444350093232"></a><p>
        In some cases, the server creates internal temporary tables
        while processing statements. Users have no direct control over
        when this occurs.
      </p><p>
        The server creates temporary tables under conditions such as
        these:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Evaluation of <a class="link" href="sql-statements.html#union" title="13.2.10.3 UNION Clause"><code class="literal">UNION</code></a>
            statements, with some exceptions described later.
          </p></li><li class="listitem"><p>
            Evaluation of some views, such those that use the
            <code class="literal">TEMPTABLE</code> algorithm,
            <a class="link" href="sql-statements.html#union" title="13.2.10.3 UNION Clause"><code class="literal">UNION</code></a>, or aggregation.
          </p></li><li class="listitem"><p>
            Evaluation of derived tables (see
            <a class="xref" href="sql-statements.html#derived-tables" title="13.2.11.8 Derived Tables">Section 13.2.11.8, “Derived Tables”</a>).
          </p></li><li class="listitem"><p>
            Evaluation of common table expressions (see
            <a class="xref" href="sql-statements.html#with" title="13.2.15 WITH (Common Table Expressions)">Section 13.2.15, “WITH (Common Table Expressions)”</a>).
          </p></li><li class="listitem"><p>
            Tables created for subquery or semijoin materialization (see
            <a class="xref" href="optimization.html#subquery-optimization" title="8.2.2 Optimizing Subqueries, Derived Tables, View References, and Common Table Expressions">Section 8.2.2, “Optimizing Subqueries, Derived Tables, View References, and Common Table
        Expressions”</a>).
          </p></li><li class="listitem"><p>
            Evaluation of statements that contain an <code class="literal">ORDER
            BY</code> clause and a different <code class="literal">GROUP
            BY</code> clause, or for which the <code class="literal">ORDER
            BY</code> or <code class="literal">GROUP BY</code> contains columns
            from tables other than the first table in the join queue.
          </p></li><li class="listitem"><p>
            Evaluation of <code class="literal">DISTINCT</code> combined with
            <code class="literal">ORDER BY</code> may require a temporary table.
          </p></li><li class="listitem"><p>
            For queries that use the <code class="literal">SQL_SMALL_RESULT</code>
            modifier, MySQL uses an in-memory temporary table, unless
            the query also contains elements (described later) that
            require on-disk storage.
          </p></li><li class="listitem"><p>
            To evaluate
            <a class="link" href="sql-statements.html#insert-select" title="13.2.6.1 INSERT ... SELECT Statement"><code class="literal">INSERT ...
            SELECT</code></a> statements that select from and insert into
            the same table, MySQL creates an internal temporary table to
            hold the rows from the
            <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>, then inserts those
            rows into the target table. See
            <a class="xref" href="sql-statements.html#insert-select" title="13.2.6.1 INSERT ... SELECT Statement">Section 13.2.6.1, “INSERT ... SELECT Statement”</a>.
          </p></li><li class="listitem"><p>
            Evaluation of multiple-table
            <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a> statements.
          </p></li><li class="listitem"><p>
            Evaluation of <a class="link" href="functions.html#function_group-concat"><code class="literal">GROUP_CONCAT()</code></a>
            or <a class="link" href="functions.html#function_count"><code class="literal">COUNT(DISTINCT)</code></a>
            expressions.
          </p></li><li class="listitem"><p>
            Evaluation of window functions (see
            <a class="xref" href="functions.html#window-functions" title="12.21 Window Functions">Section 12.21, “Window Functions”</a>) uses temporary tables as
            necessary.
</p></li></ul>
</div>
<p>
        To determine whether a statement requires a temporary table, use
        <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> and check the
        <code class="literal">Extra</code> column to see whether it says
        <code class="literal">Using temporary</code> (see
        <a class="xref" href="optimization.html#using-explain" title="8.8.1 Optimizing Queries with EXPLAIN">Section 8.8.1, “Optimizing Queries with EXPLAIN”</a>). <code class="literal">EXPLAIN</code>
        will not necessarily say <code class="literal">Using temporary</code> for
        derived or materialized temporary tables. For statements that
        use window functions, <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>
        with <code class="literal">FORMAT=JSON</code> always provides information
        about the windowing steps. If the windowing functions use
        temporary tables, it is indicated for each step.
      </p><p>
        When the server creates an internal temporary table (either in
        memory or on disk), it increments the
        <a class="link" href="server-administration.html#statvar_Created_tmp_tables"><code class="literal">Created_tmp_tables</code></a> status
        variable. If the server creates the table on disk (either
        initially or by converting an in-memory table) it increments the
        <a class="link" href="server-administration.html#statvar_Created_tmp_disk_tables"><code class="literal">Created_tmp_disk_tables</code></a> status
        variable.
      </p><p>
        Some query conditions prevent the use of an in-memory temporary
        table, in which case the server uses an on-disk table instead:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Presence of a <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a> or
            <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">TEXT</code></a> column in the table.
            However, the <code class="literal">TempTable</code> storage engine,
            which is the default storage engine for in-memory internal
            temporary tables in MySQL 8.0, supports binary
            large object types as of MySQL 8.0.13. See
            <a class="xref" href="optimization.html#internal-temporary-tables-engines" title="Internal Temporary Table Storage Engine">Internal Temporary Table Storage Engine</a>.
          </p></li><li class="listitem"><p>
            Presence of any string column with a maximum length larger
            than 512 (bytes for binary strings, characters for nonbinary
            strings) in the <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> list,
            if <a class="link" href="sql-statements.html#union" title="13.2.10.3 UNION Clause"><code class="literal">UNION</code></a> or
            <a class="link" href="sql-statements.html#union" title="13.2.10.3 UNION Clause"><code class="literal">UNION ALL</code></a>
            is used.
          </p></li><li class="listitem"><p>
            The <a class="link" href="sql-statements.html#show-columns" title="13.7.7.5 SHOW COLUMNS Statement"><code class="literal">SHOW COLUMNS</code></a> and
            <a class="link" href="sql-statements.html#describe" title="13.8.1 DESCRIBE Statement"><code class="literal">DESCRIBE</code></a> statements use
            <code class="literal">BLOB</code> as the type for some columns, thus
            the temporary table used for the results is an on-disk
            table.
</p></li></ul>
</div>
<p>
        The server does not use a temporary table for
        <a class="link" href="sql-statements.html#union" title="13.2.10.3 UNION Clause"><code class="literal">UNION</code></a> statements that meet
        certain qualifications. Instead, it retains from temporary table
        creation only the data structures necessary to perform result
        column typecasting. The table is not fully instantiated and no
        rows are written to or read from it; rows are sent directly to
        the client. The result is reduced memory and disk requirements,
        and smaller delay before the first row is sent to the client
        because the server need not wait until the last query block is
        executed. <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> and optimizer
        trace output reflects this execution strategy: The
        <code class="literal">UNION RESULT</code> query block is not present
        because that block corresponds to the part that reads from the
        temporary table.
      </p><p>
        These conditions qualify a <code class="literal">UNION</code> for
        evaluation without a temporary table:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            The union is <code class="literal">UNION ALL</code>, not
            <code class="literal">UNION</code> or <code class="literal">UNION
            DISTINCT</code>.
          </p></li><li class="listitem"><p>
            There is no global <code class="literal">ORDER BY</code> clause.
          </p></li><li class="listitem"><p>
            The union is not the top-level query block of an
            <code class="literal">{INSERT | REPLACE} ... SELECT ...</code>
            statement.
</p></li></ul>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="internal-temporary-tables-engines"></a>Internal Temporary Table Storage Engine</h4>

</div>

</div>

</div>
<p>
          An internal temporary table can be held in memory and
          processed by the <code class="literal">TempTable</code> or
          <code class="literal">MEMORY</code> storage engine, or stored on disk by
          the <code class="literal">InnoDB</code> storage engine.
</p>
<h5><a name="idm46444350020080"></a>Storage Engine for In-Memory Internal Temporary Tables</h5>
<p>
          The
          <a class="link" href="server-administration.html#sysvar_internal_tmp_mem_storage_engine"><code class="literal">internal_tmp_mem_storage_engine</code></a>
          session variable defines the storage engine for in-memory
          internal temporary tables. Permitted values are
          <code class="literal">TempTable</code> (the default) and
          <code class="literal">MEMORY</code>.
        </p><p>
          The <code class="literal">TempTable</code> storage engine provides
          efficient storage for <a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">VARCHAR</code></a>
          and <a class="link" href="data-types.html#binary-varbinary" title="11.3.3 The BINARY and VARBINARY Types"><code class="literal">VARBINARY</code></a> columns. Storage
          of other binary large object types is supported as of MySQL
          8.0.13. The <a class="link" href="server-administration.html#sysvar_temptable_max_ram"><code class="literal">temptable_max_ram</code></a>
          variable defines the maximum amount of RAM that can be
          occupied by the <code class="literal">TempTable</code> storage engine
          before it starts allocating space from disk in the form
          memory-mapped temporary files (the default) or
          <code class="literal">InnoDB</code> on-disk internal temporary tables.
          The default <a class="link" href="server-administration.html#sysvar_temptable_max_ram"><code class="literal">temptable_max_ram</code></a>
          setting is 1GiB. The
          <a class="link" href="server-administration.html#sysvar_temptable_use_mmap"><code class="literal">temptable_use_mmap</code></a> variable
          (introduced in MySQL 8.0.16) controls whether the TempTable
          storage engine uses memory-mapped files or
          <code class="literal">InnoDB</code> on-disk internal temporary tables
          when the <a class="link" href="server-administration.html#sysvar_temptable_max_ram"><code class="literal">temptable_max_ram</code></a>
          limit is exceeded. The default setting is
          <a class="link" href="server-administration.html#sysvar_temptable_use_mmap"><code class="literal">temptable_use_mmap=ON</code></a>.
</p>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
            The <a class="link" href="server-administration.html#sysvar_temptable_max_ram"><code class="literal">temptable_max_ram</code></a>
            setting does not account for the thread-local memory block
            allocated to each thread that uses the
            <code class="literal">TempTable</code> storage engine. The size of the
            thread-local memory block depends on the size of the
            thread's first memory allocation request. If the
            request is less than 1MB, which it is in most cases, the
            thread-local memory block size is 1MB. If the request is
            greater than 1MB, the thread-local memory block is
            approximately the same size as the initial memory request.
            The thread-local memory block is held in thread-local
            storage until thread exit.
</p>
</div>
<p>
          Use of memory-mapped temporary files by the
          <code class="literal">TempTable</code> storage engine as an overflow
          mechanism for in-memory temporary tables is governed by these
          rules:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Temporary files are created in the directory defined by
              the <a class="link" href="server-administration.html#sysvar_tmpdir"><code class="literal">tmpdir</code></a> variable.
            </p></li><li class="listitem"><p>
              Temporary files are deleted immediately after they are
              created and opened, and therefore do not remain visible in
              the <a class="link" href="server-administration.html#sysvar_tmpdir"><code class="literal">tmpdir</code></a> directory. The
              space occupied by temporary files is held by the operating
              system while temporary files are open. The space is
              reclaimed when temporary files are closed by the
              <code class="literal">TempTable</code> storage engine, or when the
              <code class="literal">mysqld</code> process is shut down.
            </p></li><li class="listitem"><p>
              Data is never moved between RAM and temporary files,
              within RAM, or between temporary files.
            </p></li><li class="listitem"><p>
              New data is stored in RAM if space becomes available
              within the limit defined by
              <a class="link" href="server-administration.html#sysvar_temptable_max_ram"><code class="literal">temptable_max_ram</code></a>.
              Otherwise, new data is stored in temporary files.
            </p></li><li class="listitem"><p>
              If space becomes available in RAM after some of the data
              for a table is written to temporary files, it is possible
              for the remaining table data to be stored in RAM.
</p></li></ul>
</div>
<p>
          If the <code class="literal">TempTable</code> storage engine is
          configured to use <code class="literal">InnoDB</code> on-disk internal
          temporary tables as the overflow mechanism
          (<a class="link" href="server-administration.html#sysvar_temptable_use_mmap"><code class="literal">temptable_use_mmap=OFF</code></a>), an
          in-memory table that exceeds the
          <a class="link" href="server-administration.html#sysvar_temptable_max_ram"><code class="literal">temptable_max_ram</code></a> limit is
          converted to an <code class="literal">InnoDB</code> on-disk internal
          temporary table, and any rows belonging to that table are
          moved from memory to the <code class="literal">InnoDB</code> on-disk
          internal temporary table. The
          <a class="link" href="server-administration.html#sysvar_internal_tmp_disk_storage_engine"><code class="literal">internal_tmp_disk_storage_engine</code></a>
          (removed in MySQL 8.0.16) variable setting has no affect on
          the <code class="literal">TempTable</code> storage engine overflow
          mechanism.
        </p><p>
          Consider using <code class="literal">InnoDB</code> on-disk internal
          temporary tables as the <code class="literal">TempTable</code> overflow
          mechanism if the TempTable storage engine often exceeds the
          memory limit defined by the
          <code class="literal">temptable_max_ram</code> variable and uses
          excessive space in the temporary directory for memory-mapped
          files. This may occur due to use of large internal temporary
          tables or extensive use of internal temporary tables.
          <code class="literal">InnoDB</code> on-disk internal temporary tables
          are created in session temporary tablespaces, which reside in
          the data directory by default. For more information, see
          <a class="xref" href="innodb-storage-engine.html#innodb-temporary-tablespace" title="15.6.3.5 Temporary Tablespaces">Section 15.6.3.5, “Temporary Tablespaces”</a>.
        </p><p>
          The <code class="literal">memory/temptable/physical_ram</code> and
          <code class="literal">memory/temptable/physical_disk</code> Performance
          Schema instruments can be used to monitor
          <code class="literal">TempTable</code> space allocation from memory and
          disk. <code class="literal">memory/temptable/physical_ram</code> reports
          the amount of allocated RAM.
          <code class="literal">memory/temptable/physical_disk</code> reports the
          amount of space allocated from disk when memory-mapped files
          are used as the TempTable overflow mechanism
          (<code class="literal">temptable_use_mmap=ON</code>). If the
          <code class="literal">physical_disk</code> instrument reports a value
          other than 0 and memory-mapped files are used as the TempTable
          overflow mechanism, the
          <a class="link" href="server-administration.html#sysvar_temptable_max_ram"><code class="literal">temptable_max_ram</code></a> threshold
          was reached at some point. Data can be queried in Performance
          Schema memory summary tables such as
          <a class="link" href="performance-schema.html#memory-summary-tables" title="26.12.18.10 Memory Summary Tables"><code class="literal">memory_summary_global_by_event_name</code></a>.
          See <a class="xref" href="performance-schema.html#memory-summary-tables" title="26.12.18.10 Memory Summary Tables">Section 26.12.18.10, “Memory Summary Tables”</a>.
        </p><p>
          When using the <code class="literal">MEMORY</code> storage engine for
          in-memory temporary tables, MySQL automatically converts an
          in-memory temporary table to an on-disk table if it becomes
          too large. The maximum size for in-memory temporary tables is
          defined by the <a class="link" href="server-administration.html#sysvar_tmp_table_size"><code class="literal">tmp_table_size</code></a>
          or <a class="link" href="server-administration.html#sysvar_max_heap_table_size"><code class="literal">max_heap_table_size</code></a> value,
          whichever is smaller. This differs from
          <code class="literal">MEMORY</code> tables explicitly created with
          <a class="link" href="sql-statements.html#create-table" title="13.1.20 CREATE TABLE Statement"><code class="literal">CREATE TABLE</code></a>. For such tables,
          only the <a class="link" href="server-administration.html#sysvar_max_heap_table_size"><code class="literal">max_heap_table_size</code></a>
          variable determines how large a table can grow, and there is
          no conversion to on-disk format.
</p>
<h5><a name="internal-temporary-tables-engines-disk"></a>Storage Engine for On-Disk Internal Temporary Tables</h5>
<p>
          Starting with MySQL 8.0.16, the server always uses the
          <code class="literal">InnoDB</code> storage engine for managing internal
          temporary tables on disk.
        </p><p>
          In MySQL 8.0.15 and earlier, the
          <a class="link" href="server-administration.html#sysvar_internal_tmp_disk_storage_engine"><code class="literal">internal_tmp_disk_storage_engine</code></a>
          variable was used to define the storage engine used for
          on-disk internal temporary tables. This variable was removed
          in MySQL 8.0.16, and the storage engine used for this purpose
          is no longer user-configurable.
        </p><p>
          In MySQL 8.0.15 and earlier: For common table expressions
          (CTEs), the storage engine used for on-disk internal temporary
          tables cannot be <code class="literal">MyISAM</code>. If
          <a class="link" href="server-administration.html#sysvar_internal_tmp_disk_storage_engine"><code class="literal">internal_tmp_disk_storage_engine=MYISAM</code></a>,
          an error occurs for any attempt to materialize a CTE using an
          on-disk temporary table.
        </p><p>
          In MySQL 8.0.15 and earlier: When using
          <a class="link" href="server-administration.html#sysvar_internal_tmp_disk_storage_engine"><code class="literal">internal_tmp_disk_storage_engine=INNODB</code></a>,
          queries that generate on-disk internal temporary tables that
          exceed <a class="link" href="innodb-storage-engine.html#innodb-limits" title="15.22 InnoDB Limits"><code class="literal">InnoDB</code>
          row or column limits</a> return <span class="errortext">Row size too
          large</span> or <span class="errortext">Too many columns</span>
          errors. The workaround is to set
          <a class="link" href="server-administration.html#sysvar_internal_tmp_disk_storage_engine"><code class="literal">internal_tmp_disk_storage_engine</code></a>
          to <code class="literal">MYISAM</code>.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="internal-temporary-tables-storage"></a>Internal Temporary Table Storage Format</h4>

</div>

</div>

</div>
<p>
          When in-memory internal temporary tables are managed by the
          <code class="literal">TempTable</code> storage engine, rows that include
          <code class="literal">VARCHAR</code> columns,
          <code class="literal">VARBINARY</code> columns, or other binary large
          object type columns (supported as of MySQL 8.0.13) are
          represented in memory by an array of cells, with each cell
          containing a NULL flag, the data length, and a data pointer.
          Column values are placed in consecutive order after the array,
          in a single region of memory, without padding. Each cell in
          the array uses 16 bytes of storage. The same storage format
          applies when the <code class="literal">TempTable</code> storage engine
          exceeds the <a class="link" href="server-administration.html#sysvar_temptable_max_ram"><code class="literal">temptable_max_ram</code></a>
          limit and starts allocating space from disk as memory-mapped
          files or <code class="literal">InnoDB</code> on-disk internal temporary
          tables.
        </p><p>
          When in-memory internal temporary tables are managed by the
          <code class="literal">MEMORY</code> storage engine, fixed-length row
          format is used. <code class="literal">VARCHAR</code> and
          <code class="literal">VARBINARY</code> column values are padded to the
          maximum column length, in effect storing them as
          <code class="literal">CHAR</code> and <code class="literal">BINARY</code> columns.
        </p><p>
          Previous to MySQL 8.0.16, on-disk internal temporary tables
          were managed by the <code class="literal">InnoDB</code> or
          <code class="literal">MyISAM</code> storage engine (depending on the
          <a class="link" href="server-administration.html#sysvar_internal_tmp_disk_storage_engine"><code class="literal">internal_tmp_disk_storage_engine</code></a>
          setting). Both engines store internal temporary tables using
          dynamic-width row format. Columns take only as much storage as
          needed, which reduces disk I/O, space requirements, and
          processing time compared to on-disk tables that use
          fixed-length rows. Beginning with MySQL 8.0.16,
          <code class="literal">internal_tmp_disk_storage_engine</code> is not
          supported, and internal temporary tables on disk are always
          handled by <code class="literal">InnoDB</code>.
        </p><p>
          When using the <code class="literal">MEMORY</code> storage engine,
          statements can initially create an in-memory internal
          temporary table and then convert it to an on-disk table if the
          table becomes too large. In such cases, better performance
          might be achieved by skipping the conversion and creating the
          internal temporary table on disk to begin with. The
          <a class="link" href="server-administration.html#sysvar_big_tables"><code class="literal">big_tables</code></a> variable can be
          used to force disk storage of internal temporary tables.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="database-count-limit"></a>8.4.5 Limits on Number of Databases and Tables</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444349924400"></a><a class="indexterm" name="idm46444349922928"></a><a class="indexterm" name="idm46444349921424"></a><a class="indexterm" name="idm46444349919920"></a><p>
        MySQL has no limit on the number of databases. The underlying
        file system may have a limit on the number of directories.
      </p><p>
        MySQL has no limit on the number of tables. The underlying file
        system may have a limit on the number of files that represent
        tables. Individual storage engines may impose engine-specific
        constraints. <code class="literal">InnoDB</code> permits up to 4 billion
        tables.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="table-size-limit"></a>8.4.6 Limits on Table Size</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444349915264"></a><a class="indexterm" name="idm46444349913776"></a><a class="indexterm" name="idm46444349912288"></a><a class="indexterm" name="idm46444349910800"></a><a class="indexterm" name="idm46444349909728"></a><a class="indexterm" name="idm46444349908240"></a><a class="indexterm" name="idm46444349906752"></a><p>
        The effective maximum table size for MySQL databases is usually
        determined by operating system constraints on file sizes, not by
        MySQL internal limits. For up-to-date information operating
        system file size limits, refer to the documentation specific to
        your operating system.
      </p><p>
        Windows users, please note that FAT and VFAT (FAT32) are
        <span class="emphasis"><em>not</em></span> considered suitable for production use
        with MySQL. Use NTFS instead.
      </p><p>
        If you encounter a full-table error, there are several reasons
        why it might have occurred:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            The disk might be full.
          </p></li><li class="listitem"><p>
            You are using <code class="literal">InnoDB</code> tables and have run
            out of room in an <code class="literal">InnoDB</code> tablespace file.
            The maximum tablespace size is also the maximum size for a
            table. For tablespace size limits, see
            <a class="xref" href="innodb-storage-engine.html#innodb-limits" title="15.22 InnoDB Limits">Section 15.22, “InnoDB Limits”</a>.
          </p><p>
            Generally, partitioning of tables into multiple tablespace
            files is recommended for tables larger than 1TB in size.
          </p></li><li class="listitem"><p>
            You have hit an operating system file size limit. For
            example, you are using <code class="literal">MyISAM</code> tables on
            an operating system that supports files only up to 2GB in
            size and you have hit this limit for the data file or index
            file.
          </p></li><li class="listitem"><p>
            You are using a <code class="literal">MyISAM</code> table and the
            space required for the table exceeds what is permitted by
            the internal pointer size. <code class="literal">MyISAM</code> permits
            data and index files to grow up to 256TB by default, but
            this limit can be changed up to the maximum permissible size
            of 65,536TB (256<sup>7</sup> − 1
            bytes).
          </p><p>
            If you need a <code class="literal">MyISAM</code> table that is larger
            than the default limit and your operating system supports
            large files, the <a class="link" href="sql-statements.html#create-table" title="13.1.20 CREATE TABLE Statement"><code class="literal">CREATE TABLE</code></a>
            statement supports <code class="literal">AVG_ROW_LENGTH</code> and
            <code class="literal">MAX_ROWS</code> options. See
            <a class="xref" href="sql-statements.html#create-table" title="13.1.20 CREATE TABLE Statement">Section 13.1.20, “CREATE TABLE Statement”</a>. The server uses these
            options to determine how large a table to permit.
          </p><p>
            If the pointer size is too small for an existing table, you
            can change the options with <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER
            TABLE</code></a> to increase a table's maximum
            permissible size. See <a class="xref" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement">Section 13.1.9, “ALTER TABLE Statement”</a>.
          </p><pre data-lang="sql" class="programlisting">ALTER TABLE <em class="replaceable"><code>tbl_name</code></em> MAX_ROWS=1000000000 AVG_ROW_LENGTH=<em class="replaceable"><code>nnn</code></em>;
</pre><p>
            You have to specify <code class="literal">AVG_ROW_LENGTH</code> only
            for tables with <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a> or
            <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">TEXT</code></a> columns; in this case,
            MySQL cannot optimize the space required based only on the
            number of rows.
          </p><p>
            To change the default size limit for
            <code class="literal">MyISAM</code> tables, set the
            <a class="link" href="server-administration.html#sysvar_myisam_data_pointer_size"><code class="literal">myisam_data_pointer_size</code></a>,
            which sets the number of bytes used for internal row
            pointers. The value is used to set the pointer size for new
            tables if you do not specify the <code class="literal">MAX_ROWS</code>
            option. The value of
            <a class="link" href="server-administration.html#sysvar_myisam_data_pointer_size"><code class="literal">myisam_data_pointer_size</code></a>
            can be from 2 to 7. A value of 4 permits tables up to 4GB; a
            value of 6 permits tables up to 256TB.
          </p><p>
            You can check the maximum data and index sizes by using this
            statement:
          </p><pre data-lang="sql" class="programlisting">SHOW TABLE STATUS FROM <em class="replaceable"><code>db_name</code></em> LIKE '<em class="replaceable"><code>tbl_name</code></em>';
</pre><p>
            You also can use <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk -dv
            /path/to/table-index-file</strong></span></a>. See
            <a class="xref" href="sql-statements.html#show" title="13.7.7 SHOW Statements">Section 13.7.7, “SHOW Statements”</a>, or <a class="xref" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility">Section 4.6.4, “<span class="command"><strong>myisamchk</strong></span> — MyISAM Table-Maintenance Utility”</a>.
          </p><p>
            Other ways to work around file-size limits for
            <code class="literal">MyISAM</code> tables are as follows:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                If your large table is read only, you can use
                <a class="link" href="programs.html#myisampack" title="4.6.6 myisampack — Generate Compressed, Read-Only MyISAM Tables"><span class="command"><strong>myisampack</strong></span></a> to compress it.
                <a class="link" href="programs.html#myisampack" title="4.6.6 myisampack — Generate Compressed, Read-Only MyISAM Tables"><span class="command"><strong>myisampack</strong></span></a> usually compresses a table
                by at least 50%, so you can have, in effect, much bigger
                tables. <a class="link" href="programs.html#myisampack" title="4.6.6 myisampack — Generate Compressed, Read-Only MyISAM Tables"><span class="command"><strong>myisampack</strong></span></a> also can merge
                multiple tables into a single table. See
                <a class="xref" href="programs.html#myisampack" title="4.6.6 myisampack — Generate Compressed, Read-Only MyISAM Tables">Section 4.6.6, “<span class="command"><strong>myisampack</strong></span> — Generate Compressed, Read-Only MyISAM Tables”</a>.
              </p></li><li class="listitem"><p>
                MySQL includes a <code class="literal">MERGE</code> library that
                enables you to handle a collection of
                <code class="literal">MyISAM</code> tables that have identical
                structure as a single <code class="literal">MERGE</code> table.
                See <a class="xref" href="storage-engines.html#merge-storage-engine" title="16.7 The MERGE Storage Engine">Section 16.7, “The MERGE Storage Engine”</a>.
</p></li></ul>
</div>
</li><li class="listitem"><p>
            You are using the <code class="literal">MEMORY</code>
            (<code class="literal">HEAP</code>) storage engine; in this case you
            need to increase the value of the
            <a class="link" href="server-administration.html#sysvar_max_heap_table_size"><code class="literal">max_heap_table_size</code></a> system
            variable. See <a class="xref" href="server-administration.html#server-system-variables" title="5.1.8 Server System Variables">Section 5.1.8, “Server System Variables”</a>.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="column-count-limit"></a>8.4.7 Limits on Table Column Count and Row Size</h3>

</div>

</div>

</div>
<p>
        This section describes limits on the number of columns in tables
        and the size of individual rows.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a class="xref" href="optimization.html#column-count-limits" title="Column Count Limits">Column Count Limits</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#row-size-limits" title="Row Size Limits">Row Size Limits</a></p></li></ul>
</div>

<div class="simplesect">

<div class="titlepage">
<div>

<div class="simple">
<h4 class="title"><a name="column-count-limits"></a>Column Count Limits</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444349850192"></a><a class="indexterm" name="idm46444349848688"></a><a class="indexterm" name="idm46444349847184"></a><p>
          MySQL has hard limit of 4096 columns per table, but the
          effective maximum may be less for a given table. The exact
          column limit depends on several factors:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              The maximum row size for a table constrains the number
              (and possibly size) of columns because the total length of
              all columns cannot exceed this size. See
              <a class="xref" href="optimization.html#row-size-limits" title="Row Size Limits">Row Size Limits</a>.
            </p></li><li class="listitem"><p>
              The storage requirements of individual columns constrain
              the number of columns that fit within a given maximum row
              size. Storage requirements for some data types depend on
              factors such as storage engine, storage format, and
              character set. See <a class="xref" href="data-types.html#storage-requirements" title="11.7 Data Type Storage Requirements">Section 11.7, “Data Type Storage Requirements”</a>.
            </p></li><li class="listitem"><p>
              Storage engines may impose additional restrictions that
              limit table column count. For example,
              <a class="link" href="innodb-storage-engine.html" title="Chapter 15 The InnoDB Storage Engine"><code class="literal">InnoDB</code></a> has a limit of 1017
              columns per table. See <a class="xref" href="innodb-storage-engine.html#innodb-limits" title="15.22 InnoDB Limits">Section 15.22, “InnoDB Limits”</a>.
              For information about other storage engines, see
              <a class="xref" href="storage-engines.html" title="Chapter 16 Alternative Storage Engines">Chapter 16, <i>Alternative Storage Engines</i></a>.
            </p></li><li class="listitem"><p>
              Functional key parts (see <a class="xref" href="sql-statements.html#create-index" title="13.1.15 CREATE INDEX Statement">Section 13.1.15, “CREATE INDEX Statement”</a>)
              are implemented as hidden virtual generated stored
              columns, so each functional key part in a table index
              counts against the table total column limit.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="row-size-limits"></a>Row Size Limits</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444349834608"></a><a class="indexterm" name="idm46444349833120"></a><a class="indexterm" name="idm46444349831632"></a><p>
          The maximum row size for a given table is determined by
          several factors:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              The internal representation of a MySQL table has a maximum
              row size limit of 65,535 bytes, even if the storage engine
              is capable of supporting larger rows.
              <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a> and
              <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">TEXT</code></a> columns only
              contribute 9 to 12 bytes toward the row size limit because
              their contents are stored separately from the rest of the
              row.
            </p></li><li class="listitem"><p>
              The maximum row size for an <code class="literal">InnoDB</code>
              table, which applies to data stored locally within a
              database page, is slightly less than half a page for 4KB,
              8KB, 16KB, and 32KB
              <a class="link" href="innodb-storage-engine.html#sysvar_innodb_page_size"><code class="literal">innodb_page_size</code></a>
              settings. For example, the maximum row size is slightly
              less than 8KB for the default 16KB
              <code class="literal">InnoDB</code> page size. For 64KB pages, the
              maximum row size is slightly less than 16KB. See
              <a class="xref" href="innodb-storage-engine.html#innodb-limits" title="15.22 InnoDB Limits">Section 15.22, “InnoDB Limits”</a>.
            </p><p>
              If a row containing
              <a class="link" href="glossary.html#glos_variable_length_type" title="variable-length type">variable-length
              columns</a> exceeds the <code class="literal">InnoDB</code>
              maximum row size, <code class="literal">InnoDB</code> selects
              variable-length columns for external off-page storage
              until the row fits within the <code class="literal">InnoDB</code>
              row size limit. The amount of data stored locally for
              variable-length columns that are stored off-page differs
              by row format. For more information, see
              <a class="xref" href="innodb-storage-engine.html#innodb-row-format" title="15.10 InnoDB Row Formats">Section 15.10, “InnoDB Row Formats”</a>.
            </p></li><li class="listitem"><p>
              Different storage formats use different amounts of page
              header and trailer data, which affects the amount of
              storage available for rows.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                  For information about <code class="literal">InnoDB</code> row
                  formats, see <a class="xref" href="innodb-storage-engine.html#innodb-row-format" title="15.10 InnoDB Row Formats">Section 15.10, “InnoDB Row Formats”</a>.
                </p></li><li class="listitem"><p>
                  For information about <code class="literal">MyISAM</code>
                  storage formats, see
                  <a class="xref" href="storage-engines.html#myisam-table-formats" title="16.2.3 MyISAM Table Storage Formats">Section 16.2.3, “MyISAM Table Storage Formats”</a>.
</p></li></ul>
</div>
</li></ul>
</div>
<h5><a name="idm46444349810608"></a>Row Size Limit Examples</h5>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              The MySQL maximum row size limit of 65,535 bytes is
              demonstrated in the following <code class="literal">InnoDB</code>
              and <code class="literal">MyISAM</code> examples. The limit is
              enforced regardless of storage engine, even though the
              storage engine may be capable of supporting larger rows.
            </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>CREATE TABLE t (a VARCHAR(10000), b VARCHAR(10000),</code></strong>
       <strong class="userinput"><code>c VARCHAR(10000), d VARCHAR(10000), e VARCHAR(10000),</code></strong>
       <strong class="userinput"><code>f VARCHAR(10000), g VARCHAR(6000)) ENGINE=InnoDB CHARACTER SET latin1;</code></strong>
ERROR 1118 (42000): Row size too large. The maximum row size for the used
table type, not counting BLOBs, is 65535. This includes storage overhead,
check the manual. You have to change some columns to TEXT or BLOBs
</pre><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>CREATE TABLE t (a VARCHAR(10000), b VARCHAR(10000),</code></strong>
       <strong class="userinput"><code>c VARCHAR(10000), d VARCHAR(10000), e VARCHAR(10000),</code></strong>
       <strong class="userinput"><code>f VARCHAR(10000), g VARCHAR(6000)) ENGINE=MyISAM CHARACTER SET latin1;</code></strong>
ERROR 1118 (42000): Row size too large. The maximum row size for the used
table type, not counting BLOBs, is 65535. This includes storage overhead,
check the manual. You have to change some columns to TEXT or BLOBs
</pre><p>
              In the following <code class="literal">MyISAM</code> example,
              changing a column to <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">TEXT</code></a>
              avoids the 65,535-byte row size limit and permits the
              operation to succeed because
              <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a> and
              <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">TEXT</code></a> columns only
              contribute 9 to 12 bytes toward the row size.
            </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>CREATE TABLE t (a VARCHAR(10000), b VARCHAR(10000),</code></strong>
       <strong class="userinput"><code>c VARCHAR(10000), d VARCHAR(10000), e VARCHAR(10000),</code></strong>
       <strong class="userinput"><code>f VARCHAR(10000), g TEXT(6000)) ENGINE=MyISAM CHARACTER SET latin1;</code></strong>
Query OK, 0 rows affected (0.02 sec)
</pre><p>
              The operation succeeds for an <code class="literal">InnoDB</code>
              table because changing a column to
              <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">TEXT</code></a> avoids the MySQL
              65,535-byte row size limit, and <code class="literal">InnoDB</code>
              off-page storage of variable-length columns avoids the
              <code class="literal">InnoDB</code> row size limit.
            </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>CREATE TABLE t (a VARCHAR(10000), b VARCHAR(10000),</code></strong>
       <strong class="userinput"><code>c VARCHAR(10000), d VARCHAR(10000), e VARCHAR(10000),</code></strong>
       <strong class="userinput"><code>f VARCHAR(10000), g TEXT(6000)) ENGINE=InnoDB CHARACTER SET latin1;</code></strong>
Query OK, 0 rows affected (0.02 sec)
</pre></li><li class="listitem"><p>
              Storage for variable-length columns includes length bytes,
              which are counted toward the row size. For example, a
              <a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">VARCHAR(255)
              CHARACTER SET utf8mb3</code></a> column takes two bytes to
              store the length of the value, so each value can take up
              to 767 bytes.
            </p><p>
              The statement to create table <code class="literal">t1</code>
              succeeds because the columns require 32,765 + 2 bytes and
              32,766 + 2 bytes, which falls within the maximum row size
              of 65,535 bytes:
            </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>CREATE TABLE t1</code></strong>
       <strong class="userinput"><code>(c1 VARCHAR(32765) NOT NULL, c2 VARCHAR(32766) NOT NULL)</code></strong>
       <strong class="userinput"><code>ENGINE = InnoDB CHARACTER SET latin1;</code></strong>
Query OK, 0 rows affected (0.02 sec)
</pre><p>
              The statement to create table <code class="literal">t2</code> fails
              because, although the column length is within the maximum
              length of 65,535 bytes, two additional bytes are required
              to record the length, which causes the row size to exceed
              65,535 bytes:
            </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>CREATE TABLE t2</code></strong>
       <strong class="userinput"><code>(c1 VARCHAR(65535) NOT NULL)</code></strong>
       <strong class="userinput"><code>ENGINE = InnoDB CHARACTER SET latin1;</code></strong>
ERROR 1118 (42000): Row size too large. The maximum row size for the used
table type, not counting BLOBs, is 65535. This includes storage overhead,
check the manual. You have to change some columns to TEXT or BLOBs
</pre><p>
              Reducing the column length to 65,533 or less permits the
              statement to succeed.
            </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>CREATE TABLE t2</code></strong>
       <strong class="userinput"><code>(c1 VARCHAR(65533) NOT NULL)</code></strong>
       <strong class="userinput"><code>ENGINE = InnoDB CHARACTER SET latin1;</code></strong>
Query OK, 0 rows affected (0.01 sec)
</pre></li><li class="listitem"><p>
              For <a class="link" href="storage-engines.html#myisam-storage-engine" title="16.2 The MyISAM Storage Engine"><code class="literal">MyISAM</code></a> tables,
              <code class="literal">NULL</code> columns require additional space
              in the row to record whether their values are
              <code class="literal">NULL</code>. Each <code class="literal">NULL</code>
              column takes one bit extra, rounded up to the nearest
              byte.
            </p><p>
              The statement to create table <code class="literal">t3</code> fails
              because <a class="link" href="storage-engines.html#myisam-storage-engine" title="16.2 The MyISAM Storage Engine"><code class="literal">MyISAM</code></a> requires space
              for <code class="literal">NULL</code> columns in addition to the
              space required for variable-length column length bytes,
              causing the row size to exceed 65,535 bytes:
            </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>CREATE TABLE t3</code></strong>
       <strong class="userinput"><code>(c1 VARCHAR(32765) NULL, c2 VARCHAR(32766) NULL)</code></strong>
       <strong class="userinput"><code>ENGINE = MyISAM CHARACTER SET latin1;</code></strong>
ERROR 1118 (42000): Row size too large. The maximum row size for the used
table type, not counting BLOBs, is 65535. This includes storage overhead,
check the manual. You have to change some columns to TEXT or BLOBs
</pre><p>
              For information about <a class="link" href="innodb-storage-engine.html" title="Chapter 15 The InnoDB Storage Engine"><code class="literal">InnoDB</code></a>
              <code class="literal">NULL</code> column storage, see
              <a class="xref" href="innodb-storage-engine.html#innodb-row-format" title="15.10 InnoDB Row Formats">Section 15.10, “InnoDB Row Formats”</a>.
            </p></li><li class="listitem"><p>
              <code class="literal">InnoDB</code> restricts row size (for data
              stored locally within the database page) to slightly less
              than half a database page for 4KB, 8KB, 16KB, and 32KB
              <a class="link" href="innodb-storage-engine.html#sysvar_innodb_page_size"><code class="literal">innodb_page_size</code></a>
              settings, and to slightly less than 16KB for 64KB pages.
            </p><p>
              The statement to create table <code class="literal">t4</code> fails
              because the defined columns exceed the row size limit for
              a 16KB <code class="literal">InnoDB</code> page.
            </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>CREATE TABLE t4 (</code></strong>
       <strong class="userinput"><code>c1 CHAR(255),c2 CHAR(255),c3 CHAR(255),</code></strong>
       <strong class="userinput"><code>c4 CHAR(255),c5 CHAR(255),c6 CHAR(255),</code></strong>
       <strong class="userinput"><code>c7 CHAR(255),c8 CHAR(255),c9 CHAR(255),</code></strong>
       <strong class="userinput"><code>c10 CHAR(255),c11 CHAR(255),c12 CHAR(255),</code></strong>
       <strong class="userinput"><code>c13 CHAR(255),c14 CHAR(255),c15 CHAR(255),</code></strong>
       <strong class="userinput"><code>c16 CHAR(255),c17 CHAR(255),c18 CHAR(255),</code></strong>
       <strong class="userinput"><code>c19 CHAR(255),c20 CHAR(255),c21 CHAR(255),</code></strong>
       <strong class="userinput"><code>c22 CHAR(255),c23 CHAR(255),c24 CHAR(255),</code></strong>
       <strong class="userinput"><code>c25 CHAR(255),c26 CHAR(255),c27 CHAR(255),</code></strong>
       <strong class="userinput"><code>c28 CHAR(255),c29 CHAR(255),c30 CHAR(255),</code></strong>
       <strong class="userinput"><code>c31 CHAR(255),c32 CHAR(255),c33 CHAR(255)</code></strong>
       <strong class="userinput"><code>) ENGINE=InnoDB ROW_FORMAT=DYNAMIC DEFAULT CHARSET latin1;</code></strong>
ERROR 1118 (42000): Row size too large (&gt; 8126). Changing some columns to TEXT or BLOB may help.
In current row format, BLOB prefix of 0 bytes is stored inline.
</pre></li></ul>
</div>

</div>

</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a name="optimizing-innodb"></a>8.5 Optimizing for InnoDB Tables</h2>

</div>

</div>

</div>
<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#optimizing-innodb-storage-layout">8.5.1 Optimizing Storage Layout for InnoDB Tables</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-innodb-transaction-management">8.5.2 Optimizing InnoDB Transaction Management</a></span></dt><dt><span class="section"><a href="optimization.html#innodb-performance-ro-txn">8.5.3 Optimizing InnoDB Read-Only Transactions</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-innodb-logging">8.5.4 Optimizing InnoDB Redo Logging</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-innodb-bulk-data-loading">8.5.5 Bulk Data Loading for InnoDB Tables</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-innodb-queries">8.5.6 Optimizing InnoDB Queries</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-innodb-ddl-operations">8.5.7 Optimizing InnoDB DDL Operations</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-innodb-diskio">8.5.8 Optimizing InnoDB Disk I/O</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-innodb-configuration-variables">8.5.9 Optimizing InnoDB Configuration Variables</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-innodb-many-tables">8.5.10 Optimizing InnoDB for Systems with Many Tables</a></span></dt></dl>
</div>
<a class="indexterm" name="idm46444349738928"></a><a class="indexterm" name="idm46444349737056"></a><p>
      <a class="link" href="innodb-storage-engine.html" title="Chapter 15 The InnoDB Storage Engine"><code class="literal">InnoDB</code></a> is the storage engine that
      MySQL customers typically use in production databases where
      reliability and concurrency are important.
      <code class="literal">InnoDB</code> is the default storage engine in MySQL.
      This section explains how to optimize database operations for
      <code class="literal">InnoDB</code> tables.
</p>
<div class="section">

<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="optimizing-innodb-storage-layout"></a>8.5.1 Optimizing Storage Layout for InnoDB Tables</h3>
</div>
</div>
</div>

<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Once your data reaches a stable size, or a growing table has
            increased by tens or some hundreds of megabytes, consider
            using the <code class="literal">OPTIMIZE TABLE</code> statement to
            reorganize the table and compact any wasted space. The
            reorganized tables require less disk I/O to perform full
            table scans. This is a straightforward technique that can
            improve performance when other techniques such as improving
            index usage or tuning application code are not practical.
          </p><p>
            <code class="literal">OPTIMIZE TABLE</code> copies the data part of
            the table and rebuilds the indexes. The benefits come from
            improved packing of data within indexes, and reduced
            fragmentation within the tablespaces and on disk. The
            benefits vary depending on the data in each table. You may
            find that there are significant gains for some and not for
            others, or that the gains decrease over time until you next
            optimize the table. This operation can be slow if the table
            is large or if the indexes being rebuilt do not fit into the
            buffer pool. The first run after adding a lot of data to a
            table is often much slower than later runs.
          </p></li><li class="listitem"><p>
            In <code class="literal">InnoDB</code>, having a long <code class="literal">PRIMARY
            KEY</code> (either a single column with a lengthy value,
            or several columns that form a long composite value) wastes
            a lot of disk space. The primary key value for a row is
            duplicated in all the secondary index records that point to
            the same row. (See <a class="xref" href="innodb-storage-engine.html#innodb-index-types" title="15.6.2.1 Clustered and Secondary Indexes">Section 15.6.2.1, “Clustered and Secondary Indexes”</a>.)
            Create an <code class="literal">AUTO_INCREMENT</code> column as the
            primary key if your primary key is long, or index a prefix
            of a long <code class="literal">VARCHAR</code> column instead of the
            entire column.
          </p></li><li class="listitem"><p>
            Use the <a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">VARCHAR</code></a> data type
            instead of <a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">CHAR</code></a> to store
            variable-length strings or for columns with many
            <code class="literal">NULL</code> values. A
            <a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">CHAR(<em class="replaceable"><code>N</code></em>)</code></a>
            column always takes <em class="replaceable"><code>N</code></em> characters
            to store data, even if the string is shorter or its value is
            <code class="literal">NULL</code>. Smaller tables fit better in the
            buffer pool and reduce disk I/O.
          </p><p>
            When using <code class="literal">COMPACT</code> row format (the
            default <code class="literal">InnoDB</code> format) and
            variable-length character sets, such as
            <code class="literal">utf8</code> or <code class="literal">sjis</code>,
            <a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">CHAR(<em class="replaceable"><code>N</code></em>)</code></a>
            columns occupy a variable amount of space, but still at
            least <em class="replaceable"><code>N</code></em> bytes.
          </p></li><li class="listitem"><p>
            For tables that are big, or contain lots of repetitive text
            or numeric data, consider using
            <code class="literal">COMPRESSED</code> row format. Less disk I/O is
            required to bring data into the buffer pool, or to perform
            full table scans. Before making a permanent decision,
            measure the amount of compression you can achieve by using
            <code class="literal">COMPRESSED</code> versus
            <code class="literal">COMPACT</code> row format.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="optimizing-innodb-transaction-management"></a>8.5.2 Optimizing InnoDB Transaction Management</h3>

</div>

</div>

</div>
<p>
        To optimize <code class="literal">InnoDB</code> transaction processing,
        find the ideal balance between the performance overhead of
        transactional features and the workload of your server. For
        example, an application might encounter performance issues if it
        commits thousands of times per second, and different performance
        issues if it commits only every 2-3 hours.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            The default MySQL setting <code class="literal">AUTOCOMMIT=1</code>
            can impose performance limitations on a busy database
            server. Where practical, wrap several related data change
            operations into a single transaction, by issuing
            <code class="literal">SET AUTOCOMMIT=0</code> or a <code class="literal">START
            TRANSACTION</code> statement, followed by a
            <code class="literal">COMMIT</code> statement after making all the
            changes.
          </p><p>
            <code class="literal">InnoDB</code> must flush the log to disk at each
            transaction commit if that transaction made modifications to
            the database. When each change is followed by a commit (as
            with the default autocommit setting), the I/O throughput of
            the storage device puts a cap on the number of potential
            operations per second.
          </p></li><li class="listitem"><p>
            Alternatively, for transactions that consist only of a
            single <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statement,
            turning on <code class="literal">AUTOCOMMIT</code> helps
            <code class="literal">InnoDB</code> to recognize read-only
            transactions and optimize them. See
            <a class="xref" href="optimization.html#innodb-performance-ro-txn" title="8.5.3 Optimizing InnoDB Read-Only Transactions">Section 8.5.3, “Optimizing InnoDB Read-Only Transactions”</a> for
            requirements.
          </p></li><li class="listitem"><p>
            Avoid performing rollbacks after inserting, updating, or
            deleting huge numbers of rows. If a big transaction is
            slowing down server performance, rolling it back can make
            the problem worse, potentially taking several times as long
            to perform as the original data change operations. Killing
            the database process does not help, because the rollback
            starts again on server startup.
          </p><p>
            To minimize the chance of this issue occurring:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                Increase the size of the
                <a class="link" href="glossary.html#glos_buffer_pool" title="buffer pool">buffer pool</a> so
                that all the data change changes can be cached rather
                than immediately written to disk.
              </p></li><li class="listitem"><p>
                Set
                <a class="link" href="innodb-storage-engine.html#sysvar_innodb_change_buffering"><code class="literal">innodb_change_buffering=all</code></a>
                so that update and delete operations are buffered in
                addition to inserts.
              </p></li><li class="listitem"><p>
                Consider issuing <code class="literal">COMMIT</code> statements
                periodically during the big data change operation,
                possibly breaking a single delete or update into
                multiple statements that operate on smaller numbers of
                rows.
</p></li></ul>
</div>
<p>
            To get rid of a runaway rollback once it occurs, increase
            the buffer pool so that the rollback becomes CPU-bound and
            runs fast, or kill the server and restart with
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_force_recovery"><code class="literal">innodb_force_recovery=3</code></a>, as
            explained in <a class="xref" href="innodb-storage-engine.html#innodb-recovery" title="15.18.2 InnoDB Recovery">Section 15.18.2, “InnoDB Recovery”</a>.
          </p><p>
            This issue is expected to be infrequent with the default
            setting
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_change_buffering"><code class="literal">innodb_change_buffering=all</code></a>,
            which allows update and delete operations to be cached in
            memory, making them faster to perform in the first place,
            and also faster to roll back if needed. Make sure to use
            this parameter setting on servers that process long-running
            transactions with many inserts, updates, or deletes.
          </p></li><li class="listitem"><p>
            If you can afford the loss of some of the latest committed
            transactions if a crash occurs, you can set the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_flush_log_at_trx_commit"><code class="literal">innodb_flush_log_at_trx_commit</code></a>
            parameter to 0. <code class="literal">InnoDB</code> tries to flush the
            log once per second anyway, although the flush is not
            guaranteed.
          </p></li><li class="listitem"><p>
            When rows are modified or deleted, the rows and associated
            <a class="link" href="glossary.html#glos_undo_log" title="undo log">undo logs</a> are not
            physically removed immediately, or even immediately after
            the transaction commits. The old data is preserved until
            transactions that started earlier or concurrently are
            finished, so that those transactions can access the previous
            state of modified or deleted rows. Thus, a long-running
            transaction can prevent <code class="literal">InnoDB</code> from
            purging data that was changed by a different transaction.
          </p></li><li class="listitem"><p>
            When rows are modified or deleted within a long-running
            transaction, other transactions using the
            <a class="link" href="innodb-storage-engine.html#isolevel_read-committed"><code class="literal">READ COMMITTED</code></a> and
            <a class="link" href="innodb-storage-engine.html#isolevel_repeatable-read"><code class="literal">REPEATABLE READ</code></a> isolation
            levels have to do more work to reconstruct the older data if
            they read those same rows.
          </p></li><li class="listitem"><p>
            When a long-running transaction modifies a table, queries
            against that table from other transactions do not make use
            of the <a class="link" href="glossary.html#glos_covering_index" title="covering index">covering
            index</a> technique. Queries that normally could retrieve
            all the result columns from a secondary index, instead look
            up the appropriate values from the table data.
          </p><p>
            If secondary index pages are found to have a
            <code class="literal">PAGE_MAX_TRX_ID</code> that is too new, or if
            records in the secondary index are delete-marked,
            <code class="literal">InnoDB</code> may need to look up records using
            a clustered index.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="innodb-performance-ro-txn"></a>8.5.3 Optimizing InnoDB Read-Only Transactions</h3>

</div>

</div>

</div>
<p>
        <code class="literal">InnoDB</code> can avoid the overhead associated with
        setting up the <a class="link" href="glossary.html#glos_transaction_id" title="transaction ID">transaction
        ID</a> (<code class="literal">TRX_ID</code> field) for transactions
        that are known to be read-only. A transaction ID is only needed
        for a <a class="link" href="glossary.html#glos_transaction" title="transaction">transaction</a> that
        might perform write operations or
        <a class="link" href="glossary.html#glos_locking_read" title="locking read">locking reads</a> such as
        <code class="literal">SELECT ... FOR UPDATE</code>. Eliminating
        unnecessary transaction IDs reduces the size of internal data
        structures that are consulted each time a query or data change
        statement constructs a <a class="link" href="glossary.html#glos_read_view" title="read view">read
        view</a>.
      </p><p>
        <code class="literal">InnoDB</code> detects read-only transactions when:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            The transaction is started with the
            <a class="link" href="sql-statements.html#commit" title="13.3.1 START TRANSACTION, COMMIT, and ROLLBACK Statements"><code class="literal">START TRANSACTION
            READ ONLY</code></a> statement. In this case, attempting to
            make changes to the database (for <code class="literal">InnoDB</code>,
            <code class="literal">MyISAM</code>, or other types of tables) causes
            an error, and the transaction continues in read-only state:
          </p><pre data-lang="none" class="programlisting">ERROR 1792 (25006): Cannot execute statement in a READ ONLY transaction.</pre><p>
            You can still make changes to session-specific temporary
            tables in a read-only transaction, or issue locking queries
            for them, because those changes and locks are not visible to
            any other transaction.
          </p></li><li class="listitem"><p>
            The <a class="link" href="server-administration.html#sysvar_autocommit"><code class="literal">autocommit</code></a> setting is
            turned on, so that the transaction is guaranteed to be a
            single statement, and the single statement making up the
            transaction is a <span class="quote">“<span class="quote">non-locking</span>”</span>
            <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statement. That is, a
            <code class="literal">SELECT</code> that does not use a <code class="literal">FOR
            UPDATE</code> or <code class="literal">LOCK IN SHARED MODE</code>
            clause.
          </p></li><li class="listitem"><p>
            The transaction is started without the <code class="literal">READ
            ONLY</code> option, but no updates or statements that
            explicitly lock rows have been executed yet. Until updates
            or explicit locks are required, a transaction stays in
            read-only mode.
</p></li></ul>
</div>
<p>
        Thus, for a read-intensive application such as a report
        generator, you can tune a sequence of <code class="literal">InnoDB</code>
        queries by grouping them inside
        <a class="link" href="sql-statements.html#commit" title="13.3.1 START TRANSACTION, COMMIT, and ROLLBACK Statements"><code class="literal">START TRANSACTION READ
        ONLY</code></a> and
        <a class="link" href="sql-statements.html#commit" title="13.3.1 START TRANSACTION, COMMIT, and ROLLBACK Statements"><code class="literal">COMMIT</code></a>, or by
        turning on the <a class="link" href="server-administration.html#sysvar_autocommit"><code class="literal">autocommit</code></a>
        setting before running the <code class="literal">SELECT</code> statements,
        or simply by avoiding any data change statements interspersed
        with the queries.
      </p><p>
        For information about
        <a class="link" href="sql-statements.html#commit" title="13.3.1 START TRANSACTION, COMMIT, and ROLLBACK Statements"><code class="literal">START
        TRANSACTION</code></a> and
        <a class="link" href="server-administration.html#sysvar_autocommit"><code class="literal">autocommit</code></a>, see
        <a class="xref" href="sql-statements.html#commit" title="13.3.1 START TRANSACTION, COMMIT, and ROLLBACK Statements">Section 13.3.1, “START TRANSACTION, COMMIT, and ROLLBACK Statements”</a>.
</p>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
          Transactions that qualify as auto-commit, non-locking, and
          read-only (AC-NL-RO) are kept out of certain internal
          <code class="literal">InnoDB</code> data structures and are therefore
          not listed in
          <a class="link" href="sql-statements.html#show-engine" title="13.7.7.15 SHOW ENGINE Statement"><code class="literal">SHOW ENGINE
          INNODB STATUS</code></a> output.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="optimizing-innodb-logging"></a>8.5.4 Optimizing InnoDB Redo Logging</h3>

</div>

</div>

</div>
<p>
        Consider the following guidelines for optimizing redo logging:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Make your redo log files big, even as big as the
            <a class="link" href="glossary.html#glos_buffer_pool" title="buffer pool">buffer pool</a>. When
            <code class="literal">InnoDB</code> has written the redo log files
            full, it must write the modified contents of the buffer pool
            to disk in a
            <a class="link" href="glossary.html#glos_checkpoint" title="checkpoint">checkpoint</a>. Small
            redo log files cause many unnecessary disk writes. Although
            historically big redo log files caused lengthy recovery
            times, recovery is now much faster and you can confidently
            use large redo log files.
          </p><p>
            The size and number of redo log files are configured using
            the <a class="link" href="innodb-storage-engine.html#sysvar_innodb_log_file_size"><code class="literal">innodb_log_file_size</code></a>
            and
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_log_files_in_group"><code class="literal">innodb_log_files_in_group</code></a>
            configuration options. For information about modifying an
            existing redo log file configuration, see
            <a class="xref" href="innodb-storage-engine.html#innodb-redo-log-file-reconfigure" title="Changing the Number or Size of Redo Log Files">Changing the Number or Size of Redo Log Files</a>.
          </p></li><li class="listitem"><p>
            Consider increasing the size of the
            <a class="link" href="glossary.html#glos_log_buffer" title="log buffer">log buffer</a>. A large
            log buffer enables large
            <a class="link" href="glossary.html#glos_transaction" title="transaction">transactions</a> to run
            without a need to write the log to disk before the
            transactions <a class="link" href="glossary.html#glos_commit" title="commit">commit</a>.
            Thus, if you have transactions that update, insert, or
            delete many rows, making the log buffer larger saves disk
            I/O. Log buffer size is configured using the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_log_buffer_size"><code class="literal">innodb_log_buffer_size</code></a>
            configuration option, which can be configured dynamically in
            MySQL 8.0.
          </p></li><li class="listitem"><p>
            Configure the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_log_write_ahead_size"><code class="literal">innodb_log_write_ahead_size</code></a>
            configuration option to avoid <span class="quote">“<span class="quote">read-on-write</span>”</span>.
            This option defines the write-ahead block size for the redo
            log. Set
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_log_write_ahead_size"><code class="literal">innodb_log_write_ahead_size</code></a>
            to match the operating system or file system cache block
            size. Read-on-write occurs when redo log blocks are not
            entirely cached to the operating system or file system due
            to a mismatch between write-ahead block size for the redo
            log and operating system or file system cache block size.
          </p><p>
            Valid values for
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_log_write_ahead_size"><code class="literal">innodb_log_write_ahead_size</code></a>
            are multiples of the <code class="literal">InnoDB</code> log file
            block size (2<sup>n</sup>). The minimum
            value is the <code class="literal">InnoDB</code> log file block size
            (512). Write-ahead does not occur when the minimum value is
            specified. The maximum value is equal to the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_page_size"><code class="literal">innodb_page_size</code></a> value. If
            you specify a value for
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_log_write_ahead_size"><code class="literal">innodb_log_write_ahead_size</code></a>
            that is larger than the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_page_size"><code class="literal">innodb_page_size</code></a> value, the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_log_write_ahead_size"><code class="literal">innodb_log_write_ahead_size</code></a>
            setting is truncated to the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_page_size"><code class="literal">innodb_page_size</code></a> value.
          </p><p>
            Setting the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_log_write_ahead_size"><code class="literal">innodb_log_write_ahead_size</code></a>
            value too low in relation to the operating system or file
            system cache block size results in read-on-write. Setting
            the value too high may have a slight impact on
            <code class="literal">fsync</code> performance for log file writes due
            to several blocks being written at once.
          </p></li><li class="listitem"><p>
            Optimize the use of spin delay by user threads waiting for
            flushed redo. Spin delay helps reduce latency. During
            periods of low concurrency, reducing latency may be less of
            a priority, and avoiding the use of spin delay during these
            periods may reduce energy consumption. During periods of
            high concurrency, you may want to avoid expending processing
            power on spin delay so that it can be used for other work.
            The following system variables permit setting high and low
            watermark values that define boundaries for the use of spin
            delay.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <a class="link" href="innodb-storage-engine.html#sysvar_innodb_log_wait_for_flush_spin_hwm"><code class="literal">innodb_log_wait_for_flush_spin_hwm</code></a>:
                Defines the maximum average log flush time beyond which
                user threads no longer spin while waiting for flushed
                redo. The default value is 400 microseconds.
              </p></li><li class="listitem"><p>
                <a class="link" href="innodb-storage-engine.html#sysvar_innodb_log_spin_cpu_abs_lwm"><code class="literal">innodb_log_spin_cpu_abs_lwm</code></a>:
                Defines the minimum amount of CPU usage below which user
                threads no longer spin while waiting for flushed redo.
                The value is expressed as a sum of CPU core usage. For
                example, The default value of 80 is 80% of a single CPU
                core. On a system with a multi-core processor, a value
                of 150 represents 100% usage of one CPU core plus 50%
                usage of a second CPU core.
              </p></li><li class="listitem"><p>
                <a class="link" href="innodb-storage-engine.html#sysvar_innodb_log_spin_cpu_pct_hwm"><code class="literal">innodb_log_spin_cpu_pct_hwm</code></a>:
                Defines the maximum amount of CPU usage above which user
                threads no longer spin while waiting for flushed redo.
                The value is expressed as a percentage of the combined
                total processing power of all CPU cores. The default
                value is 50%. For example, 100% usage of two CPU cores
                is 50% of the combined CPU processing power on a server
                with four CPU cores.
              </p><p>
                The
                <a class="link" href="innodb-storage-engine.html#sysvar_innodb_log_spin_cpu_pct_hwm"><code class="literal">innodb_log_spin_cpu_pct_hwm</code></a>
                configuration option respects processor affinity. For
                example, if a server has 48 cores but the
                <a class="link" href="programs.html#mysqld" title="4.3.1 mysqld — The MySQL Server"><span class="command"><strong>mysqld</strong></span></a> process is pinned to only four
                CPU cores, the other 44 CPU cores are ignored.
</p></li></ul>
</div>
</li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="optimizing-innodb-bulk-data-loading"></a>8.5.5 Bulk Data Loading for InnoDB Tables</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444349585008"></a><p>
        These performance tips supplement the general guidelines for
        fast inserts in <a class="xref" href="optimization.html#insert-optimization" title="8.2.5.1 Optimizing INSERT Statements">Section 8.2.5.1, “Optimizing INSERT Statements”</a>.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            When importing data into <code class="literal">InnoDB</code>, turn off
            autocommit mode, because it performs a log flush to disk for
            every insert. To disable autocommit during your import
            operation, surround it with
            <a class="link" href="sql-statements.html#commit" title="13.3.1 START TRANSACTION, COMMIT, and ROLLBACK Statements"><code class="literal">SET
            autocommit</code></a> and
            <a class="link" href="sql-statements.html#commit" title="13.3.1 START TRANSACTION, COMMIT, and ROLLBACK Statements"><code class="literal">COMMIT</code></a> statements:
          </p><pre data-lang="sql" class="programlisting">SET autocommit=0;<em class="replaceable"><code>... SQL import statements ...</code></em>
COMMIT;
</pre><p>
            The <a class="link" href="programs.html#mysqldump" title="4.5.4 mysqldump — A Database Backup Program"><span class="command"><strong>mysqldump</strong></span></a> option
            <a class="link" href="programs.html#option_mysqldump_opt"><code class="option">--opt</code></a> creates dump files
            that are fast to import into an <code class="literal">InnoDB</code>
            table, even without wrapping them with the
            <a class="link" href="sql-statements.html#commit" title="13.3.1 START TRANSACTION, COMMIT, and ROLLBACK Statements"><code class="literal">SET
            autocommit</code></a> and
            <a class="link" href="sql-statements.html#commit" title="13.3.1 START TRANSACTION, COMMIT, and ROLLBACK Statements"><code class="literal">COMMIT</code></a> statements.
          </p></li><li class="listitem"><p>
            If you have <code class="literal">UNIQUE</code> constraints on
            secondary keys, you can speed up table imports by
            temporarily turning off the uniqueness checks during the
            import session:
          </p><pre data-lang="sql" class="programlisting">SET unique_checks=0;<em class="replaceable"><code>... SQL import statements ...</code></em>
SET unique_checks=1;
</pre><p>
            For big tables, this saves a lot of disk I/O because
            <code class="literal">InnoDB</code> can use its change buffer to write
            secondary index records in a batch. Be certain that the data
            contains no duplicate keys.
          </p></li><li class="listitem"><p>
            If you have <code class="literal">FOREIGN KEY</code> constraints in
            your tables, you can speed up table imports by turning off
            the foreign key checks for the duration of the import
            session:
          </p><pre data-lang="sql" class="programlisting">SET foreign_key_checks=0;<em class="replaceable"><code>... SQL import statements ...</code></em>
SET foreign_key_checks=1;
</pre><p>
            For big tables, this can save a lot of disk I/O.
          </p></li><li class="listitem"><p>
            Use the multiple-row <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>
            syntax to reduce communication overhead between the client
            and the server if you need to insert many rows:
          </p><pre data-lang="sql" class="programlisting">INSERT INTO yourtable VALUES (1,2), (5,5), ...;</pre><p>
            This tip is valid for inserts into any table, not just
            <code class="literal">InnoDB</code> tables.
          </p></li><li class="listitem"><p>
            When doing bulk inserts into tables with auto-increment
            columns, set
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_autoinc_lock_mode"><code class="literal">innodb_autoinc_lock_mode</code></a> to
            2 (interleaved) instead of 1 (consecutive). See
            <a class="xref" href="innodb-storage-engine.html#innodb-auto-increment-handling" title="15.6.1.6 AUTO_INCREMENT Handling in InnoDB">Section 15.6.1.6, “AUTO_INCREMENT Handling in InnoDB”</a> for
            details.
          </p></li><li class="listitem"><p>
            When performing bulk inserts, it is faster to insert rows in
            <code class="literal">PRIMARY KEY</code> order.
            <code class="literal">InnoDB</code> tables use a
            <a class="link" href="glossary.html#glos_clustered_index" title="clustered index">clustered index</a>,
            which makes it relatively fast to use data in the order of
            the <code class="literal">PRIMARY KEY</code>. Performing bulk inserts
            in <code class="literal">PRIMARY KEY</code> order is particularly
            important for tables that do not fit entirely within the
            buffer pool.
          </p></li><li class="listitem"><p>
            For optimal performance when loading data into an
            <code class="literal">InnoDB</code> <code class="literal">FULLTEXT</code> index,
            follow this set of steps:
</p>
<div class="orderedlist">
<ol class="orderedlist" type="1"><li class="listitem"><p>
                Define a column <code class="literal">FTS_DOC_ID</code> at table
                creation time, of type <code class="literal">BIGINT UNSIGNED NOT
                NULL</code>, with a unique index named
                <code class="literal">FTS_DOC_ID_INDEX</code>. For example:
              </p><pre data-lang="sql" class="programlisting">CREATE TABLE t1 (
FTS_DOC_ID BIGINT unsigned NOT NULL AUTO_INCREMENT,
title varchar(255) NOT NULL DEFAULT '',
text mediumtext NOT NULL,
PRIMARY KEY (`FTS_DOC_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
CREATE UNIQUE INDEX FTS_DOC_ID_INDEX on t1(FTS_DOC_ID);</pre></li><li class="listitem"><p>
                Load the data into the table.
              </p></li><li class="listitem"><p>
                Create the <code class="literal">FULLTEXT</code> index after the
                data is loaded.
</p></li></ol>
</div>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">
<div class="admon-title">
Note
</div>
<p>
              When adding <code class="literal">FTS_DOC_ID</code> column at table
              creation time, ensure that the
              <code class="literal">FTS_DOC_ID</code> column is updated when the
              <code class="literal">FULLTEXT</code> indexed column is updated, as
              the <code class="literal">FTS_DOC_ID</code> must increase
              monotonically with each
              <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a> or
              <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a>. If you choose not
              to add the <code class="literal">FTS_DOC_ID</code> at table creation
              time and have <code class="literal">InnoDB</code> manage DOC IDs for
              you, <code class="literal">InnoDB</code> will add the
              <code class="literal">FTS_DOC_ID</code> as a hidden column with the
              next <a class="link" href="sql-statements.html#create-index" title="13.1.15 CREATE INDEX Statement"><code class="literal">CREATE
              FULLTEXT INDEX</code></a> call. This approach, however,
              requires a table rebuild which will impact performance.
</p>
</div>
</li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="optimizing-innodb-queries"></a>8.5.6 Optimizing InnoDB Queries</h3>

</div>

</div>

</div>
<p>
        To tune queries for <code class="literal">InnoDB</code> tables, create an
        appropriate set of indexes on each table. See
        <a class="xref" href="optimization.html#mysql-indexes" title="8.3.1 How MySQL Uses Indexes">Section 8.3.1, “How MySQL Uses Indexes”</a> for details. Follow these
        guidelines for <code class="literal">InnoDB</code> indexes:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Because each <code class="literal">InnoDB</code> table has a
            <a class="link" href="glossary.html#glos_primary_key" title="primary key">primary key</a> (whether
            you request one or not), specify a set of primary key
            columns for each table, columns that are used in the most
            important and time-critical queries.
          </p></li><li class="listitem"><p>
            Do not specify too many or too long columns in the primary
            key, because these column values are duplicated in each
            secondary index. When an index contains unnecessary data,
            the I/O to read this data and memory to cache it reduce the
            performance and scalability of the server.
          </p></li><li class="listitem"><p>
            Do not create a separate
            <a class="link" href="glossary.html#glos_secondary_index" title="secondary index">secondary index</a>
            for each column, because each query can only make use of one
            index. Indexes on rarely tested columns or columns with only
            a few different values might not be helpful for any queries.
            If you have many queries for the same table, testing
            different combinations of columns, try to create a small
            number of
            <a class="link" href="glossary.html#glos_concatenated_index" title="concatenated index">concatenated
            indexes</a> rather than a large number of single-column
            indexes. If an index contains all the columns needed for the
            result set (known as a
            <a class="link" href="glossary.html#glos_covering_index" title="covering index">covering index</a>),
            the query might be able to avoid reading the table data at
            all.
          </p></li><li class="listitem"><p>
            If an indexed column cannot contain any
            <code class="literal">NULL</code> values, declare it as <code class="literal">NOT
            NULL</code> when you create the table. The optimizer can
            better determine which index is most effective to use for a
            query, when it knows whether each column contains
            <code class="literal">NULL</code> values.
          </p></li><li class="listitem"><p>
            You can optimize single-query transactions for
            <code class="literal">InnoDB</code> tables, using the technique in
            <a class="xref" href="optimization.html#innodb-performance-ro-txn" title="8.5.3 Optimizing InnoDB Read-Only Transactions">Section 8.5.3, “Optimizing InnoDB Read-Only Transactions”</a>.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="optimizing-innodb-ddl-operations"></a>8.5.7 Optimizing InnoDB DDL Operations</h3>

</div>

</div>

</div>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Many DDL operations on tables and indexes
            (<code class="literal">CREATE</code>, <code class="literal">ALTER</code>, and
            <code class="literal">DROP</code> statements) can be performed online.
            See <a class="xref" href="innodb-storage-engine.html#innodb-online-ddl" title="15.12 InnoDB and Online DDL">Section 15.12, “InnoDB and Online DDL”</a> for details.
          </p></li><li class="listitem"><p>
            Online DDL support for adding secondary indexes means that
            you can generally speed up the process of creating and
            loading a table and associated indexes by creating the table
            without secondary indexes, then adding secondary indexes
            after the data is loaded.
          </p></li><li class="listitem"><p>
            Use <a class="link" href="sql-statements.html#truncate-table" title="13.1.37 TRUNCATE TABLE Statement"><code class="literal">TRUNCATE TABLE</code></a> to empty a
            table, not <code class="literal">DELETE FROM
            <em class="replaceable"><code>tbl_name</code></em></code>. Foreign key
            constraints can make a <code class="literal">TRUNCATE</code> statement
            work like a regular <code class="literal">DELETE</code> statement, in
            which case a sequence of commands like
            <a class="link" href="sql-statements.html#drop-table" title="13.1.32 DROP TABLE Statement"><code class="literal">DROP TABLE</code></a> and
            <a class="link" href="sql-statements.html#create-table" title="13.1.20 CREATE TABLE Statement"><code class="literal">CREATE TABLE</code></a> might be
            fastest.
          </p></li><li class="listitem"><p>
            Because the primary key is integral to the storage layout of
            each <code class="literal">InnoDB</code> table, and changing the
            definition of the primary key involves reorganizing the
            whole table, always set up the primary key as part of the
            <a class="link" href="sql-statements.html#create-table" title="13.1.20 CREATE TABLE Statement"><code class="literal">CREATE TABLE</code></a> statement, and
            plan ahead so that you do not need to
            <code class="literal">ALTER</code> or <code class="literal">DROP</code> the
            primary key afterward.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="optimizing-innodb-diskio"></a>8.5.8 Optimizing InnoDB Disk I/O</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444349492048"></a><a class="indexterm" name="idm46444349490560"></a><a class="indexterm" name="idm46444349489488"></a><a class="indexterm" name="idm46444349488416"></a><p>
        If you follow best practices for database design and tuning
        techniques for SQL operations, but your database is still slow
        due to heavy disk I/O activity, consider these disk I/O
        optimizations. If the Unix <code class="filename">top</code> tool or the
        Windows Task Manager shows that the CPU usage percentage with
        your workload is less than 70%, your workload is probably
        disk-bound.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Increase buffer pool size
          </p><p>
            When table data is cached in the <code class="literal">InnoDB</code>
            buffer pool, it can be accessed repeatedly by queries
            without requiring any disk I/O. Specify the size of the
            buffer pool with the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_buffer_pool_size"><code class="literal">innodb_buffer_pool_size</code></a>
            option. This memory area is important enough that it is
            typically recommended that
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_buffer_pool_size"><code class="literal">innodb_buffer_pool_size</code></a> is
            configured to 50 to 75 percent of system memory. For more
            information see, <a class="xref" href="optimization.html#memory-use" title="8.12.3.1 How MySQL Uses Memory">Section 8.12.3.1, “How MySQL Uses Memory”</a>.
          </p></li><li class="listitem"><p>
            Adjust the flush method
          </p><p>
            In some versions of GNU/Linux and Unix, flushing files to
            disk with the Unix <code class="literal">fsync()</code> call (which
            <code class="literal">InnoDB</code> uses by default) and similar
            methods is surprisingly slow. If database write performance
            is an issue, conduct benchmarks with the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_flush_method"><code class="literal">innodb_flush_method</code></a>
            parameter set to <code class="literal">O_DSYNC</code>.
          </p></li><li class="listitem"><p>
            Configure an fsync threshold
          </p><p>
            By default, when <code class="literal">InnoDB</code> creates a new
            data file, such as a new log file or tablespace file, the
            file is fully written to the operating system cache before
            it is flushed to disk, which can cause a large amount of
            disk write activity to occur at once. To force smaller,
            periodic flushes of data from the operating system cache,
            you can use the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_fsync_threshold"><code class="literal">innodb_fsync_threshold</code></a>
            variable to define a threshold value, in bytes. When the
            byte threshold is reached, the contents of the operating
            system cache are flushed to disk. The default value of 0
            forces the default behavior, which is to flush data to disk
            only after a file is fully written to the cache.
          </p><p>
            Specifying a threshold to force smaller, periodic flushes
            may be beneficial in cases where multiple MySQL instances
            use the same storage devices. For example, creating a new
            MySQL instance and its associated data files could cause
            large surges of disk write activity, impeding the
            performance of other MySQL instances that use the same
            storage devices. Configuring a threshold helps avoid such
            surges in write activity.
          </p></li><li class="listitem"><p>
            Use a noop or deadline I/O scheduler with native AIO on
            Linux
          </p><p>
            <code class="literal">InnoDB</code> uses the asynchronous I/O
            subsystem (native AIO) on Linux to perform read-ahead and
            write requests for data file pages. This behavior is
            controlled by the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_use_native_aio"><code class="literal">innodb_use_native_aio</code></a>
            configuration option, which is enabled by default. With
            native AIO, the type of I/O scheduler has greater influence
            on I/O performance. Generally, noop and deadline I/O
            schedulers are recommended. Conduct benchmarks to determine
            which I/O scheduler provides the best results for your
            workload and environment. For more information, see
            <a class="xref" href="innodb-storage-engine.html#innodb-linux-native-aio" title="15.8.6 Using Asynchronous I/O on Linux">Section 15.8.6, “Using Asynchronous I/O on Linux”</a>.
          </p></li><li class="listitem"><p>
            Use direct I/O on Solaris 10 for x86_64 architecture
          </p><p>
            When using the <code class="literal">InnoDB</code> storage engine on
            Solaris 10 for x86_64 architecture (AMD Opteron), use direct
            I/O for <code class="literal">InnoDB</code>-related files to avoid
            degradation of <code class="literal">InnoDB</code> performance. To use
            direct I/O for an entire UFS file system used for storing
            <code class="literal">InnoDB</code>-related files, mount it with the
            <code class="literal">forcedirectio</code> option; see
            <code class="literal">mount_ufs(1M)</code>. (The default on Solaris
            10/x86_64 is <span class="emphasis"><em>not</em></span> to use this option.)
            To apply direct I/O only to <code class="literal">InnoDB</code> file
            operations rather than the whole file system, set
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_flush_method"><code class="literal">innodb_flush_method =
            O_DIRECT</code></a>. With this setting,
            <code class="literal">InnoDB</code> calls
            <code class="literal">directio()</code> instead of
            <code class="literal">fcntl()</code> for I/O to data files (not for
            I/O to log files).
          </p></li><li class="listitem"><p>
            Use raw storage for data and log files with Solaris 2.6 or
            later
          </p><p>
            When using the <code class="literal">InnoDB</code> storage engine with
            a large
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_buffer_pool_size"><code class="literal">innodb_buffer_pool_size</code></a>
            value on any release of Solaris 2.6 and up and any platform
            (sparc/x86/x64/amd64), conduct benchmarks with
            <code class="literal">InnoDB</code> data files and log files on raw
            devices or on a separate direct I/O UFS file system, using
            the <code class="literal">forcedirectio</code> mount option as
            described previously. (It is necessary to use the mount
            option rather than setting
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_flush_method"><code class="literal">innodb_flush_method</code></a> if you
            want direct I/O for the log files.) Users of the Veritas
            file system VxFS should use the
            <code class="literal">convosync=direct</code> mount option.
          </p><p>
            Do not place other MySQL data files, such as those for
            <code class="literal">MyISAM</code> tables, on a direct I/O file
            system. Executables or libraries <span class="emphasis"><em>must
            not</em></span> be placed on a direct I/O file system.
          </p></li><li class="listitem"><p>
            Use additional storage devices
          </p><p>
            Additional storage devices could be used to set up a RAID
            configuration. For related information, see
            <a class="xref" href="optimization.html#disk-issues" title="8.12.1 Optimizing Disk I/O">Section 8.12.1, “Optimizing Disk I/O”</a>.
          </p><p>
            Alternatively, <code class="literal">InnoDB</code> tablespace data
            files and log files can be placed on different physical
            disks. For more information, refer to the following
            sections:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <a class="xref" href="innodb-storage-engine.html#innodb-init-startup-configuration" title="15.8.1 InnoDB Startup Configuration">Section 15.8.1, “InnoDB Startup Configuration”</a>
              </p></li><li class="listitem"><p>
                <a class="xref" href="innodb-storage-engine.html#innodb-create-table-external" title="15.6.1.2 Creating Tables Externally">Section 15.6.1.2, “Creating Tables Externally”</a>
              </p></li><li class="listitem"><p>
                <a class="xref" href="innodb-storage-engine.html#general-tablespaces-creating" title="Creating a General Tablespace">Creating a General Tablespace</a>
              </p></li><li class="listitem"><p>
                <a class="xref" href="innodb-storage-engine.html#innodb-migration" title="15.6.1.4 Moving or Copying InnoDB Tables">Section 15.6.1.4, “Moving or Copying InnoDB Tables”</a>
</p></li></ul>
</div>
</li><li class="listitem"><p>
            Consider non-rotational storage
          </p><p>
            Non-rotational storage generally provides better performance
            for random I/O operations; and rotational storage for
            sequential I/O operations. When distributing data and log
            files across rotational and non-rotational storage devices,
            consider the type of I/O operations that are predominantly
            performed on each file.
          </p><p>
            Random I/O-oriented files typically include
            <a class="link" href="glossary.html#glos_file_per_table" title="file-per-table">file-per-table</a>
            and <a class="link" href="glossary.html#glos_general_tablespace" title="general tablespace">general
            tablespace</a> data files,
            <a class="link" href="glossary.html#glos_undo_tablespace" title="undo tablespace">undo tablespace</a>
            files, and
            <a class="link" href="glossary.html#glos_temporary_tablespace" title="temporary tablespace">temporary
            tablespace</a> files. Sequential I/O-oriented files
            include <code class="literal">InnoDB</code>
            <a class="link" href="glossary.html#glos_system_tablespace" title="system tablespace">system
            tablespace</a> files (due to
            <a class="link" href="glossary.html#glos_doublewrite_buffer" title="doublewrite buffer">doublewrite
            buffering</a> prior to MySQL 8.0.20 and
            <a class="link" href="glossary.html#glos_change_buffer" title="change buffer">change buffering</a>),
            doublewrite files introduced in MySQL 8.0.20, and log files
            such as <a class="link" href="glossary.html#glos_binary_log" title="binary log">binary log</a>
            files and <a class="link" href="glossary.html#glos_redo_log" title="redo log">redo log</a>
            files.
          </p><p>
            Review settings for the following configuration options when
            using non-rotational storage:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <a class="link" href="innodb-storage-engine.html#sysvar_innodb_checksum_algorithm"><code class="literal">innodb_checksum_algorithm</code></a>
              </p><p>
                The <code class="literal">crc32</code> option uses a faster
                checksum algorithm and is recommended for fast storage
                systems.
              </p></li><li class="listitem"><p>
                <a class="link" href="innodb-storage-engine.html#sysvar_innodb_flush_neighbors"><code class="literal">innodb_flush_neighbors</code></a>
              </p><p>
                Optimizes I/O for rotational storage devices. Disable it
                for non-rotational storage or a mix of rotational and
                non-rotational storage. It is disabled by default.
              </p></li><li class="listitem"><p>
                <a class="link" href="innodb-storage-engine.html#sysvar_innodb_idle_flush_pct"><code class="literal">innodb_idle_flush_pct</code></a>
              </p><p>
                Permits placing a limit on page flushing during idle
                periods, which can help extend the life of
                non-rotational storage devices. Introduced in MySQL
                8.0.18.
              </p></li><li class="listitem"><p>
                <a class="link" href="innodb-storage-engine.html#sysvar_innodb_io_capacity"><code class="literal">innodb_io_capacity</code></a>
              </p><p>
                The default setting of 200 is generally sufficient for a
                lower-end non-rotational storage device. For higher-end,
                bus-attached devices, consider a higher setting such as
                1000.
              </p></li><li class="listitem"><p>
                <a class="link" href="innodb-storage-engine.html#sysvar_innodb_io_capacity_max"><code class="literal">innodb_io_capacity_max</code></a>
              </p><p>
                The default value of 2000 is intended for workloads that
                use non-rotational storage. For a high-end, bus-attached
                non-rotational storage device, consider a higher setting
                such as 2500.
              </p></li><li class="listitem"><p>
                <a class="link" href="innodb-storage-engine.html#sysvar_innodb_log_compressed_pages"><code class="literal">innodb_log_compressed_pages</code></a>
              </p><p>
                If redo logs are on non-rotational storage, consider
                disabling this option to reduce logging. See
                <a class="link" href="optimization.html#innodb-disable-log-compressed-pages">Disable
                logging of compressed pages</a>.
              </p></li><li class="listitem"><p>
                <a class="link" href="innodb-storage-engine.html#sysvar_innodb_log_file_size"><code class="literal">innodb_log_file_size</code></a>
              </p><p>
                If redo logs are on non-rotational storage, configure
                this option to maximize caching and write combining.
              </p></li><li class="listitem"><p>
                <a class="link" href="innodb-storage-engine.html#sysvar_innodb_page_size"><code class="literal">innodb_page_size</code></a>
              </p><p>
                Consider using a page size that matches the internal
                sector size of the disk. Early-generation SSD devices
                often have a 4KB sector size. Some newer devices have a
                16KB sector size. The default <code class="literal">InnoDB</code>
                page size is 16KB. Keeping the page size close to the
                storage device block size minimizes the amount of
                unchanged data that is rewritten to disk.
              </p></li><li class="listitem"><p>
                <a class="link" href="replication.html#sysvar_binlog_row_image"><code class="literal">binlog_row_image</code></a>
              </p><p>
                If binary logs are on non-rotational storage and all
                tables have primary keys, consider setting this option
                to <code class="literal">minimal</code> to reduce logging.
</p></li></ul>
</div>
<p>
            Ensure that TRIM support is enabled for your operating
            system. It is typically enabled by default.
          </p></li><li class="listitem"><p>
            Increase I/O capacity to avoid backlogs
          </p><p>
            If throughput drops periodically because of
            <code class="literal">InnoDB</code>
            <a class="link" href="glossary.html#glos_checkpoint" title="checkpoint">checkpoint</a>
            operations, consider increasing the value of the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_io_capacity"><code class="literal">innodb_io_capacity</code></a>
            configuration option. Higher values cause more frequent
            <a class="link" href="glossary.html#glos_flush" title="flush">flushing</a>, avoiding the
            backlog of work that can cause dips in throughput.
          </p></li><li class="listitem"><p>
            Lower I/O capacity if flushing does not fall behind
          </p><p>
            If the system is not falling behind with
            <code class="literal">InnoDB</code>
            <a class="link" href="glossary.html#glos_flush" title="flush">flushing</a> operations,
            consider lowering the value of the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_io_capacity"><code class="literal">innodb_io_capacity</code></a>
            configuration option. Typically, you keep this option value
            as low as practical, but not so low that it causes periodic
            drops in throughput as mentioned in the preceding bullet. In
            a typical scenario where you could lower the option value,
            you might see a combination like this in the output from
            <a class="link" href="sql-statements.html#show-engine" title="13.7.7.15 SHOW ENGINE Statement"><code class="literal">SHOW ENGINE
            INNODB STATUS</code></a>:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                History list length low, below a few thousand.
              </p></li><li class="listitem"><p>
                Insert buffer merges close to rows inserted.
              </p></li><li class="listitem"><p>
                Modified pages in buffer pool consistently well below
                <a class="link" href="innodb-storage-engine.html#sysvar_innodb_max_dirty_pages_pct"><code class="literal">innodb_max_dirty_pages_pct</code></a>
                of the buffer pool. (Measure at a time when the server
                is not doing bulk inserts; it is normal during bulk
                inserts for the modified pages percentage to rise
                significantly.)
              </p></li><li class="listitem"><p>
                <code class="literal">Log sequence number - Last checkpoint</code>
                is at less than 7/8 or ideally less than 6/8 of the
                total size of the <code class="literal">InnoDB</code>
                <a class="link" href="glossary.html#glos_log_file" title="log file">log files</a>.
</p></li></ul>
</div>
</li><li class="listitem"><p>
            Store system tablespace files on Fusion-io devices
          </p><p>
            You can take advantage of a doublewrite buffer-related I/O
            optimization by storing the files that contain the
            doublewrite storage area on Fusion-io devices that support
            atomic writes. (Prior to MySQL 8.0.20, the doublewrite
            buffer storage are resides in the system tablespace data
            files. As of MySQL 8.0.20, the storage area resides in
            doublewrite files. See
            <a class="xref" href="innodb-storage-engine.html#innodb-doublewrite-buffer" title="15.6.4 Doublewrite Buffer">Section 15.6.4, “Doublewrite Buffer”</a>.) When
            doublewrite storage area files are placed on Fusion-io
            devices that support atomic writes, the doublewrite buffer
            is automatically disabled and Fusion-io atomic writes are
            used for all data files. This feature is only supported on
            Fusion-io hardware and is only enabled for Fusion-io NVMFS
            on Linux. To take full advantage of this feature, an
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_flush_method"><code class="literal">innodb_flush_method</code></a> setting
            of <code class="literal">O_DIRECT</code> is recommended.
</p>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
              Because the doublewrite buffer setting is global, the
              doublewrite buffer is also disabled for data files that do
              not reside on Fusion-io hardware.
</p>
</div>
</li><li class="listitem"><p><a name="innodb-disable-log-compressed-pages"></a>
            Disable logging of compressed pages
          </p><p>
            When using the <code class="literal">InnoDB</code> table
            <a class="link" href="glossary.html#glos_compression" title="compression">compression</a> feature,
            images of re-compressed
            <a class="link" href="glossary.html#glos_page" title="page">pages</a> are written to the
            <a class="link" href="glossary.html#glos_redo_log" title="redo log">redo log</a> when changes
            are made to compressed data. This behavior is controlled by
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_log_compressed_pages"><code class="literal">innodb_log_compressed_pages</code></a>,
            which is enabled by default to prevent corruption that can
            occur if a different version of the <code class="literal">zlib</code>
            compression algorithm is used during recovery. If you are
            certain that the <code class="literal">zlib</code> version will not
            change, disable
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_log_compressed_pages"><code class="literal">innodb_log_compressed_pages</code></a>
            to reduce redo log generation for workloads that modify
            compressed data.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="optimizing-innodb-configuration-variables"></a>8.5.9 Optimizing InnoDB Configuration Variables</h3>

</div>

</div>

</div>
<p>
        Different settings work best for servers with light, predictable
        loads, versus servers that are running near full capacity all
        the time, or that experience spikes of high activity.
      </p><p>
        Because the <code class="literal">InnoDB</code> storage engine performs
        many of its optimizations automatically, many performance-tuning
        tasks involve monitoring to ensure that the database is
        performing well, and changing configuration options when
        performance drops. See
        <a class="xref" href="innodb-storage-engine.html#innodb-performance-schema" title="15.16 InnoDB Integration with MySQL Performance Schema">Section 15.16, “InnoDB Integration with MySQL Performance Schema”</a> for information
        about detailed <code class="literal">InnoDB</code> performance monitoring.
      </p><p>
        The main configuration steps you can perform include:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Controlling the types of data change operations for which
            <code class="literal">InnoDB</code> buffers the changed data, to avoid
            frequent small disk writes. See
            <a class="xref" href="innodb-storage-engine.html#innodb-change-buffer-configuration" title="Configuring Change Buffering">Configuring Change Buffering</a>.
            Because the default is to buffer all types of data change
            operations, only change this setting if you need to reduce
            the amount of buffering.
          </p></li><li class="listitem"><p>
            Turning the adaptive hash indexing feature on and off using
            the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_adaptive_hash_index"><code class="literal">innodb_adaptive_hash_index</code></a>
            option. See <a class="xref" href="innodb-storage-engine.html#innodb-adaptive-hash" title="15.5.3 Adaptive Hash Index">Section 15.5.3, “Adaptive Hash Index”</a> for more
            information. You might change this setting during periods of
            unusual activity, then restore it to its original setting.
          </p></li><li class="listitem"><p>
            Setting a limit on the number of concurrent threads that
            <code class="literal">InnoDB</code> processes, if context switching is
            a bottleneck. See
            <a class="xref" href="innodb-storage-engine.html#innodb-performance-thread_concurrency" title="15.8.4 Configuring Thread Concurrency for InnoDB">Section 15.8.4, “Configuring Thread Concurrency for InnoDB”</a>.
          </p></li><li class="listitem"><p>
            Controlling the amount of prefetching that
            <code class="literal">InnoDB</code> does with its read-ahead
            operations. When the system has unused I/O capacity, more
            read-ahead can improve the performance of queries. Too much
            read-ahead can cause periodic drops in performance on a
            heavily loaded system. See
            <a class="xref" href="innodb-storage-engine.html#innodb-performance-read_ahead" title="15.8.3.4 Configuring InnoDB Buffer Pool Prefetching (Read-Ahead)">Section 15.8.3.4, “Configuring InnoDB Buffer Pool Prefetching (Read-Ahead)”</a>.
          </p></li><li class="listitem"><p>
            Increasing the number of background threads for read or
            write operations, if you have a high-end I/O subsystem that
            is not fully utilized by the default values. See
            <a class="xref" href="innodb-storage-engine.html#innodb-performance-multiple_io_threads" title="15.8.5 Configuring the Number of Background InnoDB I/O Threads">Section 15.8.5, “Configuring the Number of Background InnoDB I/O Threads”</a>.
          </p></li><li class="listitem"><p>
            Controlling how much I/O <code class="literal">InnoDB</code> performs
            in the background. See
            <a class="xref" href="innodb-storage-engine.html#innodb-configuring-io-capacity" title="15.8.7 Configuring InnoDB I/O Capacity">Section 15.8.7, “Configuring InnoDB I/O Capacity”</a>. You might
            scale back this setting if you observe periodic drops in
            performance.
          </p></li><li class="listitem"><p>
            Controlling the algorithm that determines when
            <code class="literal">InnoDB</code> performs certain types of
            background writes. See
            <a class="xref" href="innodb-storage-engine.html#innodb-buffer-pool-flushing" title="15.8.3.5 Configuring Buffer Pool Flushing">Section 15.8.3.5, “Configuring Buffer Pool Flushing”</a>. The algorithm
            works for some types of workloads but not others, so you
            might disable this feature if you observe periodic drops in
            performance.
          </p></li><li class="listitem"><p>
            Taking advantage of multicore processors and their cache
            memory configuration, to minimize delays in context
            switching. See
            <a class="xref" href="innodb-storage-engine.html#innodb-performance-spin_lock_polling" title="15.8.8 Configuring Spin Lock Polling">Section 15.8.8, “Configuring Spin Lock Polling”</a>.
          </p></li><li class="listitem"><p>
            Preventing one-time operations such as table scans from
            interfering with the frequently accessed data stored in the
            <code class="literal">InnoDB</code> buffer cache. See
            <a class="xref" href="innodb-storage-engine.html#innodb-performance-midpoint_insertion" title="15.8.3.3 Making the Buffer Pool Scan Resistant">Section 15.8.3.3, “Making the Buffer Pool Scan Resistant”</a>.
          </p></li><li class="listitem"><p>
            Adjusting log files to a size that makes sense for
            reliability and crash recovery. <code class="literal">InnoDB</code>
            log files have often been kept small to avoid long startup
            times after a crash. Optimizations introduced in MySQL 5.5
            speed up certain steps of the crash
            <a class="link" href="glossary.html#glos_crash_recovery" title="crash recovery">recovery</a> process.
            In particular, scanning the
            <a class="link" href="glossary.html#glos_redo_log" title="redo log">redo log</a> and applying
            the redo log are faster due to improved algorithms for
            memory management. If you have kept your log files
            artificially small to avoid long startup times, you can now
            consider increasing log file size to reduce the I/O that
            occurs due recycling of redo log records.
          </p></li><li class="listitem"><p>
            Configuring the size and number of instances for the
            <code class="literal">InnoDB</code> buffer pool, especially important
            for systems with multi-gigabyte buffer pools. See
            <a class="xref" href="innodb-storage-engine.html#innodb-multiple-buffer-pools" title="15.8.3.2 Configuring Multiple Buffer Pool Instances">Section 15.8.3.2, “Configuring Multiple Buffer Pool Instances”</a>.
          </p></li><li class="listitem"><p>
            Increasing the maximum number of concurrent transactions,
            which dramatically improves scalability for the busiest
            databases. See <a class="xref" href="innodb-storage-engine.html#innodb-undo-logs" title="15.6.6 Undo Logs">Section 15.6.6, “Undo Logs”</a>.
          </p></li><li class="listitem"><p>
            Moving purge operations (a type of garbage collection) into
            a background thread. See
            <a class="xref" href="innodb-storage-engine.html#innodb-purge-configuration" title="15.8.9 Purge Configuration">Section 15.8.9, “Purge Configuration”</a>. To effectively
            measure the results of this setting, tune the other
            I/O-related and thread-related configuration settings first.
          </p></li><li class="listitem"><p>
            Reducing the amount of switching that
            <code class="literal">InnoDB</code> does between concurrent threads,
            so that SQL operations on a busy server do not queue up and
            form a <span class="quote">“<span class="quote">traffic jam</span>”</span>. Set a value for the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_thread_concurrency"><code class="literal">innodb_thread_concurrency</code></a>
            option, up to approximately 32 for a high-powered modern
            system. Increase the value for the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_concurrency_tickets"><code class="literal">innodb_concurrency_tickets</code></a>
            option, typically to 5000 or so. This combination of options
            sets a cap on the number of threads that
            <code class="literal">InnoDB</code> processes at any one time, and
            allows each thread to do substantial work before being
            swapped out, so that the number of waiting threads stays low
            and operations can complete without excessive context
            switching.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="optimizing-innodb-many-tables"></a>8.5.10 Optimizing InnoDB for Systems with Many Tables</h3>

</div>

</div>

</div>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            If you have configured
            <a class="link" href="innodb-storage-engine.html#innodb-statistics-estimation" title="15.8.10.2 Configuring Non-Persistent Optimizer Statistics Parameters">non-persistent
            optimizer statistics</a> (a non-default configuration),
            <code class="literal">InnoDB</code> computes index
            <a class="link" href="glossary.html#glos_cardinality" title="cardinality">cardinality</a> values
            for a table the first time that table is accessed after
            startup, instead of storing such values in the table. This
            step can take significant time on systems that partition the
            data into many tables. Since this overhead only applies to
            the initial table open operation, to <span class="quote">“<span class="quote">warm up</span>”</span>
            a table for later use, access it immediately after startup
            by issuing a statement such as <code class="literal">SELECT 1 FROM
            <em class="replaceable"><code>tbl_name</code></em> LIMIT 1</code>.
          </p><p>
            Optimizer statistics are persisted to disk by default,
            enabled by the
            <a class="link" href="innodb-storage-engine.html#sysvar_innodb_stats_persistent"><code class="literal">innodb_stats_persistent</code></a>
            configuration option. For information about persistent
            optimizer statistics, see
            <a class="xref" href="innodb-storage-engine.html#innodb-persistent-stats" title="15.8.10.1 Configuring Persistent Optimizer Statistics Parameters">Section 15.8.10.1, “Configuring Persistent Optimizer Statistics Parameters”</a>.
</p></li></ul>
</div>

</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a name="optimizing-myisam"></a>8.6 Optimizing for MyISAM Tables</h2>

</div>

</div>

</div>
<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#optimizing-queries-myisam">8.6.1 Optimizing MyISAM Queries</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-myisam-bulk-data-loading">8.6.2 Bulk Data Loading for MyISAM Tables</a></span></dt><dt><span class="section"><a href="optimization.html#repair-table-optimization">8.6.3 Optimizing REPAIR TABLE Statements</a></span></dt></dl>
</div>
<a class="indexterm" name="idm46444349306528"></a><p>
      The <a class="link" href="storage-engines.html#myisam-storage-engine" title="16.2 The MyISAM Storage Engine"><code class="literal">MyISAM</code></a> storage engine performs
      best with read-mostly data or with low-concurrency operations,
      because table locks limit the ability to perform simultaneous
      updates. In MySQL, <a class="link" href="innodb-storage-engine.html" title="Chapter 15 The InnoDB Storage Engine"><code class="literal">InnoDB</code></a> is the
      default storage engine rather than <code class="literal">MyISAM</code>.
</p>
<div class="section">

<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="optimizing-queries-myisam"></a>8.6.1 Optimizing MyISAM Queries</h3>
</div>
</div>
</div>
<p>
        Some general tips for speeding up queries on
        <code class="literal">MyISAM</code> tables:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            To help MySQL better optimize queries, use
            <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE TABLE</code></a> or run
            <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk --analyze</strong></span></a> on a table after it
            has been loaded with data. This updates a value for each
            index part that indicates the average number of rows that
            have the same value. (For unique indexes, this is always 1.)
            MySQL uses this to decide which index to choose when you
            join two tables based on a nonconstant expression. You can
            check the result from the table analysis by using
            <code class="literal">SHOW INDEX FROM
            <em class="replaceable"><code>tbl_name</code></em></code> and examining
            the <code class="literal">Cardinality</code> value. <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk
            --description --verbose</strong></span></a> shows index distribution
            information.
          </p></li><li class="listitem"><p>
            To sort an index and data according to an index, use
            <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk --sort-index --sort-records=1</strong></span></a>
            (assuming that you want to sort on index 1). This is a good
            way to make queries faster if you have a unique index from
            which you want to read all rows in order according to the
            index. The first time you sort a large table this way, it
            may take a long time.
          </p></li><li class="listitem"><p>
            Try to avoid complex <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>
            queries on <code class="literal">MyISAM</code> tables that are updated
            frequently, to avoid problems with table locking that occur
            due to contention between readers and writers.
          </p></li><li class="listitem"><p>
            <code class="literal">MyISAM</code> supports concurrent inserts: If a
            table has no free blocks in the middle of the data file, you
            can <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a> new rows into it
            at the same time that other threads are reading from the
            table. If it is important to be able to do this, consider
            using the table in ways that avoid deleting rows. Another
            possibility is to run <a class="link" href="sql-statements.html#optimize-table" title="13.7.3.4 OPTIMIZE TABLE Statement"><code class="literal">OPTIMIZE
            TABLE</code></a> to defragment the table after you have
            deleted a lot of rows from it. This behavior is altered by
            setting the
            <a class="link" href="server-administration.html#sysvar_concurrent_insert"><code class="literal">concurrent_insert</code></a> variable.
            You can force new rows to be appended (and therefore permit
            concurrent inserts), even in tables that have deleted rows.
            See <a class="xref" href="optimization.html#concurrent-inserts" title="8.11.3 Concurrent Inserts">Section 8.11.3, “Concurrent Inserts”</a>.
          </p></li><li class="listitem"><p>
            For <code class="literal">MyISAM</code> tables that change frequently,
            try to avoid all variable-length columns
            (<a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">VARCHAR</code></a>,
            <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a>, and
            <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">TEXT</code></a>). The table uses dynamic
            row format if it includes even a single variable-length
            column. See <a class="xref" href="storage-engines.html" title="Chapter 16 Alternative Storage Engines">Chapter 16, <i>Alternative Storage Engines</i></a>.
          </p></li><li class="listitem"><p>
            It is normally not useful to split a table into different
            tables just because the rows become large. In accessing a
            row, the biggest performance hit is the disk seek needed to
            find the first byte of the row. After finding the data, most
            modern disks can read the entire row fast enough for most
            applications. The only cases where splitting up a table
            makes an appreciable difference is if it is a
            <code class="literal">MyISAM</code> table using dynamic row format
            that you can change to a fixed row size, or if you very
            often need to scan the table but do not need most of the
            columns. See <a class="xref" href="storage-engines.html" title="Chapter 16 Alternative Storage Engines">Chapter 16, <i>Alternative Storage Engines</i></a>.
          </p></li><li class="listitem"><p>
            Use <code class="literal">ALTER TABLE ... ORDER BY
            <em class="replaceable"><code>expr1</code></em>,
            <em class="replaceable"><code>expr2</code></em>, ...</code> if you
            usually retrieve rows in
            <code class="literal"><em class="replaceable"><code>expr1</code></em>,
            <em class="replaceable"><code>expr2</code></em>, ...</code> order. By
            using this option after extensive changes to the table, you
            may be able to get higher performance.
          </p></li><li class="listitem"><p>
            If you often need to calculate results such as counts based
            on information from a lot of rows, it may be preferable to
            introduce a new table and update the counter in real time.
            An update of the following form is very fast:
          </p><pre data-lang="sql" class="programlisting">UPDATE <em class="replaceable"><code>tbl_name</code></em> SET <em class="replaceable"><code>count_col</code></em>=<em class="replaceable"><code>count_col</code></em>+1 WHERE <em class="replaceable"><code>key_col</code></em>=<em class="replaceable"><code>constant</code></em>;
</pre><p>
            This is very important when you use MySQL storage engines
            such as <code class="literal">MyISAM</code> that has only table-level
            locking (multiple readers with single writers). This also
            gives better performance with most database systems, because
            the row locking manager in this case has less to do.
          </p></li><li class="listitem"><p>
            Use <a class="link" href="sql-statements.html#optimize-table" title="13.7.3.4 OPTIMIZE TABLE Statement"><code class="literal">OPTIMIZE TABLE</code></a>
            periodically to avoid fragmentation with dynamic-format
            <code class="literal">MyISAM</code> tables. See
            <a class="xref" href="storage-engines.html#myisam-table-formats" title="16.2.3 MyISAM Table Storage Formats">Section 16.2.3, “MyISAM Table Storage Formats”</a>.
          </p></li><li class="listitem"><p>
            Declaring a <code class="literal">MyISAM</code> table with the
            <code class="literal">DELAY_KEY_WRITE=1</code> table option makes
            index updates faster because they are not flushed to disk
            until the table is closed. The downside is that if something
            kills the server while such a table is open, you must ensure
            that the table is okay by running the server with the
            <a class="link" href="server-administration.html#sysvar_myisam_recover_options"><code class="literal">myisam_recover_options</code></a>
            system variable set, or by running
            <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> before restarting the server.
            (However, even in this case, you should not lose anything by
            using <code class="literal">DELAY_KEY_WRITE</code>, because the key
            information can always be generated from the data rows.)
          </p></li><li class="listitem"><p>
            Strings are automatically prefix- and end-space compressed
            in <code class="literal">MyISAM</code> indexes. See
            <a class="xref" href="sql-statements.html#create-index" title="13.1.15 CREATE INDEX Statement">Section 13.1.15, “CREATE INDEX Statement”</a>.
          </p></li><li class="listitem"><p>
            You can increase performance by caching queries or answers
            in your application and then executing many inserts or
            updates together. Locking the table during this operation
            ensures that the index cache is only flushed once after all
            updates.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="optimizing-myisam-bulk-data-loading"></a>8.6.2 Bulk Data Loading for MyISAM Tables</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444349247136"></a><p>
        These performance tips supplement the general guidelines for
        fast inserts in <a class="xref" href="optimization.html#insert-optimization" title="8.2.5.1 Optimizing INSERT Statements">Section 8.2.5.1, “Optimizing INSERT Statements”</a>.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            For a <code class="literal">MyISAM</code> table, you can use
            concurrent inserts to add rows at the same time that
            <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements are
            running, if there are no deleted rows in middle of the data
            file. See <a class="xref" href="optimization.html#concurrent-inserts" title="8.11.3 Concurrent Inserts">Section 8.11.3, “Concurrent Inserts”</a>.
          </p></li><li class="listitem"><p>
            With some extra work, it is possible to make
            <a class="link" href="sql-statements.html#load-data" title="13.2.7 LOAD DATA Statement"><code class="literal">LOAD DATA</code></a> run even faster for
            a <code class="literal">MyISAM</code> table when the table has many
            indexes. Use the following procedure:
</p>
<div class="orderedlist">
<ol class="orderedlist" type="1"><li class="listitem"><p>
                Execute a <a class="link" href="sql-statements.html#flush-tables"><code class="literal">FLUSH TABLES</code></a>
                statement or a <a class="link" href="programs.html#mysqladmin" title="4.5.2 mysqladmin — Client for Administering a MySQL Server"><span class="command"><strong>mysqladmin
                flush-tables</strong></span></a> command.
              </p></li><li class="listitem"><p>
                Use <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk --keys-used=0 -rq
                <em class="replaceable"><code>/path/to/db/tbl_name</code></em></strong></span></a>
                to remove all use of indexes for the table.
              </p></li><li class="listitem"><p>
                Insert data into the table with
                <a class="link" href="sql-statements.html#load-data" title="13.2.7 LOAD DATA Statement"><code class="literal">LOAD DATA</code></a>. This does not
                update any indexes and therefore is very fast.
              </p></li><li class="listitem"><p>
                If you intend only to read from the table in the future,
                use <a class="link" href="programs.html#myisampack" title="4.6.6 myisampack — Generate Compressed, Read-Only MyISAM Tables"><span class="command"><strong>myisampack</strong></span></a> to compress it. See
                <a class="xref" href="storage-engines.html#compressed-format" title="16.2.3.3 Compressed Table Characteristics">Section 16.2.3.3, “Compressed Table Characteristics”</a>.
              </p></li><li class="listitem"><p>
                Re-create the indexes with <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk -rq
                <em class="replaceable"><code>/path/to/db/tbl_name</code></em></strong></span></a>.
                This creates the index tree in memory before writing it
                to disk, which is much faster than updating the index
                during <a class="link" href="sql-statements.html#load-data" title="13.2.7 LOAD DATA Statement"><code class="literal">LOAD DATA</code></a> because
                it avoids lots of disk seeks. The resulting index tree
                is also perfectly balanced.
              </p></li><li class="listitem"><p>
                Execute a <a class="link" href="sql-statements.html#flush-tables"><code class="literal">FLUSH TABLES</code></a>
                statement or a <a class="link" href="programs.html#mysqladmin" title="4.5.2 mysqladmin — Client for Administering a MySQL Server"><span class="command"><strong>mysqladmin
                flush-tables</strong></span></a> command.
</p></li></ol>
</div>
<p>
            <a class="link" href="sql-statements.html#load-data" title="13.2.7 LOAD DATA Statement"><code class="literal">LOAD DATA</code></a> performs the
            preceding optimization automatically if the
            <code class="literal">MyISAM</code> table into which you insert data
            is empty. The main difference between automatic optimization
            and using the procedure explicitly is that you can let
            <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> allocate much more temporary
            memory for the index creation than you might want the server
            to allocate for index re-creation when it executes the
            <a class="link" href="sql-statements.html#load-data" title="13.2.7 LOAD DATA Statement"><code class="literal">LOAD DATA</code></a> statement.
          </p><p>
            You can also disable or enable the nonunique indexes for a
            <code class="literal">MyISAM</code> table by using the following
            statements rather than <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a>. If you
            use these statements, you can skip the
            <a class="link" href="sql-statements.html#flush-tables"><code class="literal">FLUSH TABLES</code></a> operations:
          </p><pre data-lang="sql" class="programlisting">ALTER TABLE <em class="replaceable"><code>tbl_name</code></em> DISABLE KEYS;
ALTER TABLE <em class="replaceable"><code>tbl_name</code></em> ENABLE KEYS;
</pre></li><li class="listitem"><p>
            To speed up <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a> operations
            that are performed with multiple statements for
            nontransactional tables, lock your tables:
          </p><pre data-lang="sql" class="programlisting">LOCK TABLES a WRITE;
INSERT INTO a VALUES (1,23),(2,34),(4,33);
INSERT INTO a VALUES (8,26),(6,29);
...
UNLOCK TABLES;</pre><p>
            This benefits performance because the index buffer is
            flushed to disk only once, after all
            <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a> statements have
            completed. Normally, there would be as many index buffer
            flushes as there are <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>
            statements. Explicit locking statements are not needed if
            you can insert all rows with a single
            <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>.
          </p><p>
            Locking also lowers the total time for multiple-connection
            tests, although the maximum wait time for individual
            connections might go up because they wait for locks. Suppose
            that five clients attempt to perform inserts simultaneously
            as follows:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                Connection 1 does 1000 inserts
              </p></li><li class="listitem"><p>
                Connections 2, 3, and 4 do 1 insert
              </p></li><li class="listitem"><p>
                Connection 5 does 1000 inserts
</p></li></ul>
</div>
<p>
            If you do not use locking, connections 2, 3, and 4 finish
            before 1 and 5. If you use locking, connections 2, 3, and 4
            probably do not finish before 1 or 5, but the total time
            should be about 40% faster.
          </p><p>
            <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>,
            <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a>, and
            <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a> operations are very
            fast in MySQL, but you can obtain better overall performance
            by adding locks around everything that does more than about
            five successive inserts or updates. If you do very many
            successive inserts, you could do a <a class="link" href="sql-statements.html#lock-tables" title="13.3.6 LOCK TABLES and UNLOCK TABLES Statements"><code class="literal">LOCK
            TABLES</code></a> followed by an
            <a class="link" href="sql-statements.html#lock-tables" title="13.3.6 LOCK TABLES and UNLOCK TABLES Statements"><code class="literal">UNLOCK
            TABLES</code></a> once in a while (each 1,000 rows or so) to
            permit other threads to access table. This would still
            result in a nice performance gain.
          </p><p>
            <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a> is still much slower
            for loading data than <a class="link" href="sql-statements.html#load-data" title="13.2.7 LOAD DATA Statement"><code class="literal">LOAD
            DATA</code></a>, even when using the strategies just
            outlined.
          </p></li><li class="listitem"><p>
            To increase performance for <code class="literal">MyISAM</code>
            tables, for both <a class="link" href="sql-statements.html#load-data" title="13.2.7 LOAD DATA Statement"><code class="literal">LOAD DATA</code></a>
            and <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>, enlarge the key
            cache by increasing the
            <a class="link" href="server-administration.html#sysvar_key_buffer_size"><code class="literal">key_buffer_size</code></a> system
            variable. See <a class="xref" href="server-administration.html#server-configuration" title="5.1.1 Configuring the Server">Section 5.1.1, “Configuring the Server”</a>.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="repair-table-optimization"></a>8.6.3 Optimizing REPAIR TABLE Statements</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444349178624"></a><p>
        <a class="link" href="sql-statements.html#repair-table" title="13.7.3.5 REPAIR TABLE Statement"><code class="literal">REPAIR TABLE</code></a> for
        <code class="literal">MyISAM</code> tables is similar to using
        <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> for repair operations, and some of
        the same performance optimizations apply:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> has variables that control
            memory allocation. You may be able to its improve
            performance by setting these variables, as described in
            <a class="xref" href="programs.html#myisamchk-memory" title="4.6.4.6 myisamchk Memory Usage">Section 4.6.4.6, “myisamchk Memory Usage”</a>.
          </p></li><li class="listitem"><p>
            For <a class="link" href="sql-statements.html#repair-table" title="13.7.3.5 REPAIR TABLE Statement"><code class="literal">REPAIR TABLE</code></a>, the same
            principle applies, but because the repair is done by the
            server, you set server system variables instead of
            <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> variables. Also, in addition to
            setting memory-allocation variables, increasing the
            <a class="link" href="server-administration.html#sysvar_myisam_max_sort_file_size"><code class="literal">myisam_max_sort_file_size</code></a>
            system variable increases the likelihood that the repair
            will use the faster filesort method and avoid the slower
            repair by key cache method. Set the variable to the maximum
            file size for your system, after checking to be sure that
            there is enough free space to hold a copy of the table
            files. The free space must be available in the file system
            containing the original table files.
</p></li></ul>
</div>
<p>
        Suppose that a <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> table-repair
        operation is done using the following options to set its
        memory-allocation variables:
      </p><pre data-lang="terminal" class="programlisting">--key_buffer_size=128M --myisam_sort_buffer_size=256M
--read_buffer_size=64M --write_buffer_size=64M</pre><p>
        Some of those <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> variables correspond
        to server system variables:
</p>
<div class="informaltable">
<table summary="myisamchk variables and corresponding server system variables."><col width="50%"><col width="50%"><thead><tr>
            <th scope="col"><a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> Variable</th>
            <th scope="col">System Variable</th>
          </tr></thead><tbody><tr>
            <td scope="row"><code class="literal">key_buffer_size</code></td>
            <td><a class="link" href="server-administration.html#sysvar_key_buffer_size"><code class="literal">key_buffer_size</code></a></td>
          </tr><tr>
            <td scope="row"><code class="literal">myisam_sort_buffer_size</code></td>
            <td><a class="link" href="server-administration.html#sysvar_myisam_sort_buffer_size"><code class="literal">myisam_sort_buffer_size</code></a></td>
          </tr><tr>
            <td scope="row"><code class="literal">read_buffer_size</code></td>
            <td><a class="link" href="server-administration.html#sysvar_read_buffer_size"><code class="literal">read_buffer_size</code></a></td>
          </tr><tr>
            <td scope="row"><code class="literal">write_buffer_size</code></td>
            <td>none</td>
</tr></tbody></table>
</div>
<p>
        Each of the server system variables can be set at runtime, and
        some of them
        (<a class="link" href="server-administration.html#sysvar_myisam_sort_buffer_size"><code class="literal">myisam_sort_buffer_size</code></a>,
        <a class="link" href="server-administration.html#sysvar_read_buffer_size"><code class="literal">read_buffer_size</code></a>) have a
        session value in addition to a global value. Setting a session
        value limits the effect of the change to your current session
        and does not affect other users. Changing a global-only variable
        (<a class="link" href="server-administration.html#sysvar_key_buffer_size"><code class="literal">key_buffer_size</code></a>,
        <a class="link" href="server-administration.html#sysvar_myisam_max_sort_file_size"><code class="literal">myisam_max_sort_file_size</code></a>)
        affects other users as well. For
        <a class="link" href="server-administration.html#sysvar_key_buffer_size"><code class="literal">key_buffer_size</code></a>, you must take
        into account that the buffer is shared with those users. For
        example, if you set the <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a>
        <code class="literal">key_buffer_size</code> variable to 128MB, you could
        set the corresponding
        <a class="link" href="server-administration.html#sysvar_key_buffer_size"><code class="literal">key_buffer_size</code></a> system variable
        larger than that (if it is not already set larger), to permit
        key buffer use by activity in other sessions. However, changing
        the global key buffer size invalidates the buffer, causing
        increased disk I/O and slowdown for other sessions. An
        alternative that avoids this problem is to use a separate key
        cache, assign to it the indexes from the table to be repaired,
        and deallocate it when the repair is complete. See
        <a class="xref" href="optimization.html#multiple-key-caches" title="8.10.2.2 Multiple Key Caches">Section 8.10.2.2, “Multiple Key Caches”</a>.
      </p><p>
        Based on the preceding remarks, a <a class="link" href="sql-statements.html#repair-table" title="13.7.3.5 REPAIR TABLE Statement"><code class="literal">REPAIR
        TABLE</code></a> operation can be done as follows to use settings
        similar to the <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> command. Here a
        separate 128MB key buffer is allocated and the file system is
        assumed to permit a file size of at least 100GB.
      </p><pre data-lang="sql" class="programlisting">SET SESSION myisam_sort_buffer_size = 256*1024*1024;
SET SESSION read_buffer_size = 64*1024*1024;
SET GLOBAL myisam_max_sort_file_size = 100*1024*1024*1024;
SET GLOBAL repair_cache.key_buffer_size = 128*1024*1024;
CACHE INDEX <em class="replaceable"><code>tbl_name</code></em> IN repair_cache;
LOAD INDEX INTO CACHE <em class="replaceable"><code>tbl_name</code></em>;
REPAIR TABLE <em class="replaceable"><code>tbl_name</code></em> ;
SET GLOBAL repair_cache.key_buffer_size = 0;
</pre><p>
        If you intend to change a global variable but want to do so only
        for the duration of a <a class="link" href="sql-statements.html#repair-table" title="13.7.3.5 REPAIR TABLE Statement"><code class="literal">REPAIR
        TABLE</code></a> operation to minimally affect other users, save
        its value in a user variable and restore it afterward. For
        example:
      </p><pre data-lang="sql" class="programlisting">SET @old_myisam_sort_buffer_size = @@GLOBAL.myisam_max_sort_file_size;
SET GLOBAL myisam_max_sort_file_size = 100*1024*1024*1024;
REPAIR TABLE tbl_name ;
SET GLOBAL myisam_max_sort_file_size = @old_myisam_max_sort_file_size;</pre><p>
        The system variables that affect <a class="link" href="sql-statements.html#repair-table" title="13.7.3.5 REPAIR TABLE Statement"><code class="literal">REPAIR
        TABLE</code></a> can be set globally at server startup if you
        want the values to be in effect by default. For example, add
        these lines to the server <code class="filename">my.cnf</code> file:
      </p><pre data-lang="init" class="programlisting">[mysqld]
myisam_sort_buffer_size=256M
key_buffer_size=1G
myisam_max_sort_file_size=100G</pre><p>
        These settings do not include
        <a class="link" href="server-administration.html#sysvar_read_buffer_size"><code class="literal">read_buffer_size</code></a>. Setting
        <a class="link" href="server-administration.html#sysvar_read_buffer_size"><code class="literal">read_buffer_size</code></a> globally to a
        large value does so for all sessions and can cause performance
        to suffer due to excessive memory allocation for a server with
        many simultaneous sessions.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a name="optimizing-memory-tables"></a>8.7 Optimizing for MEMORY Tables</h2>

</div>

</div>

</div>
<a class="indexterm" name="idm46444349111424"></a><p>
      Consider using <code class="literal">MEMORY</code> tables for noncritical
      data that is accessed often, and is read-only or rarely updated.
      Benchmark your application against equivalent
      <code class="literal">InnoDB</code> or <code class="literal">MyISAM</code> tables
      under a realistic workload, to confirm that any additional
      performance is worth the risk of losing data, or the overhead of
      copying data from a disk-based table at application start.
    </p><p>
      For best performance with <code class="literal">MEMORY</code> tables,
      examine the kinds of queries against each table, and specify the
      type to use for each associated index, either a B-tree index or a
      hash index. On the <a class="link" href="sql-statements.html#create-index" title="13.1.15 CREATE INDEX Statement"><code class="literal">CREATE INDEX</code></a>
      statement, use the clause <code class="literal">USING BTREE</code> or
      <code class="literal">USING HASH</code>. B-tree indexes are fast for queries
      that do greater-than or less-than comparisons through operators
      such as <code class="literal">&gt;</code> or <code class="literal">BETWEEN</code>.
      Hash indexes are only fast for queries that look up single values
      through the <code class="literal">=</code> operator, or a restricted set of
      values through the <code class="literal">IN</code> operator. For why
      <code class="literal">USING BTREE</code> is often a better choice than the
      default <code class="literal">USING HASH</code>, see
      <a class="xref" href="optimization.html#table-scan-avoidance" title="8.2.1.23 Avoiding Full Table Scans">Section 8.2.1.23, “Avoiding Full Table Scans”</a>. For implementation details
      of the different types of <code class="literal">MEMORY</code> indexes, see
      <a class="xref" href="optimization.html#index-btree-hash" title="8.3.9 Comparison of B-Tree and Hash Indexes">Section 8.3.9, “Comparison of B-Tree and Hash Indexes”</a>.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a name="execution-plan-information"></a>8.8 Understanding the Query Execution Plan</h2>

</div>

</div>

</div>
<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#using-explain">8.8.1 Optimizing Queries with EXPLAIN</a></span></dt><dt><span class="section"><a href="optimization.html#explain-output">8.8.2 EXPLAIN Output Format</a></span></dt><dt><span class="section"><a href="optimization.html#explain-extended">8.8.3 Extended EXPLAIN Output Format</a></span></dt><dt><span class="section"><a href="optimization.html#explain-for-connection">8.8.4 Obtaining Execution Plan Information for a Named Connection</a></span></dt><dt><span class="section"><a href="optimization.html#estimating-performance">8.8.5 Estimating Query Performance</a></span></dt></dl>
</div>
<p>
      Depending on the details of your tables, columns, indexes, and the
      conditions in your <code class="literal">WHERE</code> clause, the MySQL
      optimizer considers many techniques to efficiently perform the
      lookups involved in an SQL query. A query on a huge table can be
      performed without reading all the rows; a join involving several
      tables can be performed without comparing every combination of
      rows. The set of operations that the optimizer chooses to perform
      the most efficient query is called the <span class="quote">“<span class="quote">query execution
      plan</span>”</span>, also known as the
      <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> plan. Your goals are to
      recognize the aspects of the
      <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> plan that indicate a query
      is optimized well, and to learn the SQL syntax and indexing
      techniques to improve the plan if you see some inefficient
      operations.
</p>
<div class="section">

<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="using-explain"></a>8.8.1 Optimizing Queries with EXPLAIN</h3>
</div>
</div>
</div>
<a class="indexterm" name="idm46444349089120"></a><a class="indexterm" name="idm46444349088080"></a><p>
        The <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> statement provides
        information about how MySQL executes statements:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> works with
            <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>,
            <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a>,
            <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>,
            <a class="link" href="sql-statements.html#replace" title="13.2.9 REPLACE Statement"><code class="literal">REPLACE</code></a>, and
            <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a> statements.
          </p></li><li class="listitem"><p>
            When <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> is used with an
            explainable statement, MySQL displays information from the
            optimizer about the statement execution plan. That is, MySQL
            explains how it would process the statement, including
            information about how tables are joined and in which order.
            For information about using
            <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> to obtain execution
            plan information, see <a class="xref" href="optimization.html#explain-output" title="8.8.2 EXPLAIN Output Format">Section 8.8.2, “EXPLAIN Output Format”</a>.
          </p></li><li class="listitem"><p>
            When <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> is used with
            <code class="literal">FOR CONNECTION
            <em class="replaceable"><code>connection_id</code></em></code> rather
            than an explainable statement, it displays the execution
            plan for the statement executing in the named connection.
            See <a class="xref" href="optimization.html#explain-for-connection" title="8.8.4 Obtaining Execution Plan Information for a Named Connection">Section 8.8.4, “Obtaining Execution Plan Information for a Named Connection”</a>.
          </p></li><li class="listitem"><p>
            For <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements,
            <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> produces additional
            execution plan information that can be displayed using
            <a class="link" href="sql-statements.html#show-warnings" title="13.7.7.40 SHOW WARNINGS Statement"><code class="literal">SHOW WARNINGS</code></a>. See
            <a class="xref" href="optimization.html#explain-extended" title="8.8.3 Extended EXPLAIN Output Format">Section 8.8.3, “Extended EXPLAIN Output Format”</a>.
          </p></li><li class="listitem"><p>
            <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> is useful for
            examining queries involving partitioned tables. See
            <a class="xref" href="partitioning.html#partitioning-info" title="23.3.5 Obtaining Information About Partitions">Section 23.3.5, “Obtaining Information About Partitions”</a>.
          </p></li><li class="listitem"><p>
            The <code class="literal">FORMAT</code> option can be used to select
            the output format. <code class="literal">TRADITIONAL</code> presents
            the output in tabular format. This is the default if no
            <code class="literal">FORMAT</code> option is present.
            <code class="literal">JSON</code> format displays the information in
            JSON format.
</p></li></ul>
</div>
<p>
        <a class="indexterm" name="idm46444349054800"></a>

        With the help of <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>, you can
        see where you should add indexes to tables so that the statement
        executes faster by using indexes to find rows. You can also use
        <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> to check whether the
        optimizer joins the tables in an optimal order. To give a hint
        to the optimizer to use a join order corresponding to the order
        in which the tables are named in a
        <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statement, begin the
        statement with <code class="literal">SELECT STRAIGHT_JOIN</code> rather
        than just <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>. (See
        <a class="xref" href="sql-statements.html#select" title="13.2.10 SELECT Statement">Section 13.2.10, “SELECT Statement”</a>.) However,
        <code class="literal">STRAIGHT_JOIN</code> may prevent indexes from being
        used because it disables semijoin transformations. See
        <a class="xref" href="optimization.html#semijoins" title="8.2.2.1 Optimizing IN and EXISTS Subquery Predicates with Semijoin Transformations">Section 8.2.2.1, “Optimizing IN and EXISTS Subquery Predicates with Semijoin
          Transformations”</a>.
      </p><p>
        The optimizer trace may sometimes provide information
        complementary to that of <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>.
        However, the optimizer trace format and content are subject to
        change between versions. For details, see
        <a class="ulink" href="https://dev.mysql.com/doc/internals/en/optimizer-tracing.html" target="_top">MySQL
        Internals: Tracing the Optimizer</a>.
      </p><p>
        If you have a problem with indexes not being used when you
        believe that they should be, run <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE
        TABLE</code></a> to update table statistics, such as cardinality
        of keys, that can affect the choices the optimizer makes. See
        <a class="xref" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement">Section 13.7.3.1, “ANALYZE TABLE Statement”</a>.
</p>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> can also be used to
          obtain information about the columns in a table.
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN
          <em class="replaceable"><code>tbl_name</code></em></code></a> is synonymous
          with <code class="literal">DESCRIBE
          <em class="replaceable"><code>tbl_name</code></em></code> and
          <code class="literal">SHOW COLUMNS FROM
          <em class="replaceable"><code>tbl_name</code></em></code>. For more
          information, see <a class="xref" href="sql-statements.html#describe" title="13.8.1 DESCRIBE Statement">Section 13.8.1, “DESCRIBE Statement”</a>, and
          <a class="xref" href="sql-statements.html#show-columns" title="13.7.7.5 SHOW COLUMNS Statement">Section 13.7.7.5, “SHOW COLUMNS Statement”</a>.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="explain-output"></a>8.8.2 EXPLAIN Output Format</h3>

</div>

</div>

</div>
<p>
        The <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> statement provides
        information about how MySQL executes statements.
        <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> works with
        <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>,
        <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a>,
        <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>,
        <a class="link" href="sql-statements.html#replace" title="13.2.9 REPLACE Statement"><code class="literal">REPLACE</code></a>, and
        <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a> statements.
      </p><p>
        <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> returns a row of
        information for each table used in the
        <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statement. It lists the
        tables in the output in the order that MySQL would read them
        while processing the statement. MySQL resolves all joins using a
        nested-loop join method. This means that MySQL reads a row from
        the first table, and then finds a matching row in the second
        table, the third table, and so on. When all tables are
        processed, MySQL outputs the selected columns and backtracks
        through the table list until a table is found for which there
        are more matching rows. The next row is read from this table and
        the process continues with the next table.
</p>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
          MySQL Workbench has a Visual Explain capability that provides a
          visual representation of
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output. See
          <a class="ulink" href="https://dev.mysql.com/doc/workbench/en/wb-tutorial-visual-explain-dbt3.html" target="_top">Tutorial: Using Explain to Improve Query Performance</a>.
</p>
</div>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a class="xref" href="optimization.html#explain-output-columns" title="EXPLAIN Output Columns">EXPLAIN Output Columns</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#explain-join-types" title="EXPLAIN Join Types">EXPLAIN Join Types</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#explain-extra-information" title="EXPLAIN Extra Information">EXPLAIN Extra Information</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#explain-output-interpretation" title="EXPLAIN Output Interpretation">EXPLAIN Output Interpretation</a></p></li></ul>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="explain-output-columns"></a>EXPLAIN Output Columns</h4>

</div>

</div>

</div>
<p>
          This section describes the output columns produced by
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>. Later sections provide
          additional information about the
          <a class="link" href="optimization.html#explain-join-types" title="EXPLAIN Join Types"><code class="literal">type</code></a>
          and
          <a class="link" href="optimization.html#explain-extra-information" title="EXPLAIN Extra Information"><code class="literal">Extra</code></a>
          columns.
        </p><p>
          Each output row from <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>
          provides information about one table. Each row contains the
          values summarized in
          <a class="xref" href="optimization.html#explain-output-column-table" title="Table 8.1 EXPLAIN Output Columns">Table 8.1, “EXPLAIN Output Columns”</a>, and described
          in more detail following the table. Column names are shown in
          the table's first column; the second column provides the
          equivalent property name shown in the output when
          <code class="literal">FORMAT=JSON</code> is used.
</p>
<div class="table">
<a name="explain-output-column-table"></a><p class="title"><b>Table 8.1 EXPLAIN Output Columns</b></p>
<div class="table-contents">
<table summary="Output columns produced by the EXPLAIN statement."><col width="25%"><col width="25%"><col width="50%"><thead><tr>
              <th scope="col">Column</th>
              <th scope="col">JSON Name</th>
              <th scope="col">Meaning</th>
            </tr></thead><tbody><tr>
              <td scope="row"><a class="link" href="optimization.html#explain_id"><code class="literal">id</code></a></td>
              <td><code class="literal">select_id</code></td>
              <td>The <code class="literal">SELECT</code> identifier</td>
            </tr><tr>
              <td scope="row"><a class="link" href="optimization.html#explain_select_type"><code class="literal">select_type</code></a></td>
              <td>None</td>
              <td>The <code class="literal">SELECT</code> type</td>
            </tr><tr>
              <td scope="row"><a class="link" href="optimization.html#explain_table"><code class="literal">table</code></a></td>
              <td><code class="literal">table_name</code></td>
              <td>The table for the output row</td>
            </tr><tr>
              <td scope="row"><a class="link" href="optimization.html#explain_partitions"><code class="literal">partitions</code></a></td>
              <td><code class="literal">partitions</code></td>
              <td>The matching partitions</td>
            </tr><tr>
              <td scope="row"><a class="link" href="optimization.html#explain_type"><code class="literal">type</code></a></td>
              <td><code class="literal">access_type</code></td>
              <td>The join type</td>
            </tr><tr>
              <td scope="row"><a class="link" href="optimization.html#explain_possible_keys"><code class="literal">possible_keys</code></a></td>
              <td><code class="literal">possible_keys</code></td>
              <td>The possible indexes to choose</td>
            </tr><tr>
              <td scope="row"><a class="link" href="optimization.html#explain_key"><code class="literal">key</code></a></td>
              <td><code class="literal">key</code></td>
              <td>The index actually chosen</td>
            </tr><tr>
              <td scope="row"><a class="link" href="optimization.html#explain_key_len"><code class="literal">key_len</code></a></td>
              <td><code class="literal">key_length</code></td>
              <td>The length of the chosen key</td>
            </tr><tr>
              <td scope="row"><a class="link" href="optimization.html#explain_ref"><code class="literal">ref</code></a></td>
              <td><code class="literal">ref</code></td>
              <td>The columns compared to the index</td>
            </tr><tr>
              <td scope="row"><a class="link" href="optimization.html#explain_rows"><code class="literal">rows</code></a></td>
              <td><code class="literal">rows</code></td>
              <td>Estimate of rows to be examined</td>
            </tr><tr>
              <td scope="row"><a class="link" href="optimization.html#explain_filtered"><code class="literal">filtered</code></a></td>
              <td><code class="literal">filtered</code></td>
              <td>Percentage of rows filtered by table condition</td>
            </tr><tr>
              <td scope="row"><a class="link" href="optimization.html#explain_extra"><code class="literal">Extra</code></a></td>
              <td>None</td>
              <td>Additional information</td>
</tr></tbody></table>
</div>

</div>
<br class="table-break">
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
            JSON properties which are <code class="literal">NULL</code> are not
            displayed in JSON-formatted <code class="literal">EXPLAIN</code>
            output.
</p>
</div>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a name="explain_id"></a>
              <code class="literal">id</code> (JSON name:
              <code class="literal">select_id</code>)
            </p><p>
              The <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> identifier. This
              is the sequential number of the
              <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> within the query.
              The value can be <code class="literal">NULL</code> if the row refers
              to the union result of other rows. In this case, the
              <code class="literal">table</code> column shows a value like
              <code class="literal">&lt;union<em class="replaceable"><code>M</code></em>,<em class="replaceable"><code>N</code></em>&gt;</code>
              to indicate that the row refers to the union of the rows
              with <code class="literal">id</code> values of
              <em class="replaceable"><code>M</code></em> and
              <em class="replaceable"><code>N</code></em>.
            </p></li><li class="listitem"><p><a name="explain_select_type"></a>
              <code class="literal">select_type</code> (JSON name: none)
            </p><p>
              The type of <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>, which
              can be any of those shown in the following table. A
              JSON-formatted <code class="literal">EXPLAIN</code> exposes the
              <code class="literal">SELECT</code> type as a property of a
              <code class="literal">query_block</code>, unless it is
              <code class="literal">SIMPLE</code> or <code class="literal">PRIMARY</code>.
              The JSON names (where applicable) are also shown in the
              table.
</p>
<div class="informaltable">
<table summary="select_type values and the meaning of each value."><col width="25%"><col width="25%"><col width="50%"><thead><tr>
                  <th scope="col"><code class="literal">select_type</code> Value</th>
                  <th scope="col">JSON Name</th>
                  <th scope="col">Meaning</th>
                </tr></thead><tbody><tr>
                  <td scope="row"><code class="literal">SIMPLE</code></td>
                  <td>None</td>
                  <td>Simple <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> (not using
                    <a class="link" href="sql-statements.html#union" title="13.2.10.3 UNION Clause"><code class="literal">UNION</code></a> or subqueries)</td>
                </tr><tr>
                  <td scope="row"><code class="literal">PRIMARY</code></td>
                  <td>None</td>
                  <td>Outermost <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a></td>
                </tr><tr>
                  <td scope="row"><a class="link" href="sql-statements.html#union" title="13.2.10.3 UNION Clause"><code class="literal">UNION</code></a></td>
                  <td>None</td>
                  <td>Second or later <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statement in a
                    <a class="link" href="sql-statements.html#union" title="13.2.10.3 UNION Clause"><code class="literal">UNION</code></a></td>
                </tr><tr>
                  <td scope="row"><code class="literal">DEPENDENT UNION</code></td>
                  <td><code class="literal">dependent</code> (<code class="literal">true</code>)</td>
                  <td>Second or later <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statement in a
                    <a class="link" href="sql-statements.html#union" title="13.2.10.3 UNION Clause"><code class="literal">UNION</code></a>, dependent on
                    outer query</td>
                </tr><tr>
                  <td scope="row"><code class="literal">UNION RESULT</code></td>
                  <td><code class="literal">union_result</code></td>
                  <td>Result of a <a class="link" href="sql-statements.html#union" title="13.2.10.3 UNION Clause"><code class="literal">UNION</code></a>.</td>
                </tr><tr>
                  <td scope="row"><a class="link" href="optimization.html#optimizer-hints-subquery" title="Subquery Optimizer Hints"><code class="literal">SUBQUERY</code></a></td>
                  <td>None</td>
                  <td>First <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> in subquery</td>
                </tr><tr>
                  <td scope="row"><code class="literal">DEPENDENT SUBQUERY</code></td>
                  <td><code class="literal">dependent</code> (<code class="literal">true</code>)</td>
                  <td>First <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> in subquery, dependent on
                    outer query</td>
                </tr><tr>
                  <td scope="row"><code class="literal">DERIVED</code></td>
                  <td>None</td>
                  <td>Derived table</td>
                </tr><tr>
                  <td scope="row"><code class="literal">DEPENDENT DERIVED</code></td>
                  <td><code class="literal">dependent</code> (<code class="literal">true</code>)</td>
                  <td>Derived table dependent on another table</td>
                </tr><tr>
                  <td scope="row"><code class="literal">MATERIALIZED</code></td>
                  <td><code class="literal">materialized_from_subquery</code></td>
                  <td>Materialized subquery</td>
                </tr><tr>
                  <td scope="row"><code class="literal">UNCACHEABLE SUBQUERY</code></td>
                  <td><code class="literal">cacheable</code> (<code class="literal">false</code>)</td>
                  <td>A subquery for which the result cannot be cached and must be
                    re-evaluated for each row of the outer query</td>
                </tr><tr>
                  <td scope="row"><code class="literal">UNCACHEABLE UNION</code></td>
                  <td><code class="literal">cacheable</code> (<code class="literal">false</code>)</td>
                  <td>The second or later select in a <a class="link" href="sql-statements.html#union" title="13.2.10.3 UNION Clause"><code class="literal">UNION</code></a>
                    that belongs to an uncacheable subquery (see
                    <code class="literal">UNCACHEABLE SUBQUERY</code>)</td>
</tr></tbody></table>
</div>
<p>
              <code class="literal">DEPENDENT</code> typically signifies the use
              of a correlated subquery. See
              <a class="xref" href="sql-statements.html#correlated-subqueries" title="13.2.11.7 Correlated Subqueries">Section 13.2.11.7, “Correlated Subqueries”</a>.
            </p><p>
              <code class="literal">DEPENDENT SUBQUERY</code> evaluation differs
              from <code class="literal">UNCACHEABLE SUBQUERY</code> evaluation.
              For <code class="literal">DEPENDENT SUBQUERY</code>, the subquery is
              re-evaluated only once for each set of different values of
              the variables from its outer context. For
              <code class="literal">UNCACHEABLE SUBQUERY</code>, the subquery is
              re-evaluated for each row of the outer context.
            </p><p>
              When you specify <code class="literal">FORMAT=JSON</code> with
              <code class="literal">EXPLAIN</code>, the output has no single
              property directly equivalent to
              <code class="literal">select_type</code>; the
              <code class="literal">query_block</code> property corresponds to a
              given <code class="literal">SELECT</code>. Properties equivalent to
              most of the <code class="literal">SELECT</code> subquery types just
              shown are available (an example being
              <code class="literal">materialized_from_subquery</code> for
              <code class="literal">MATERIALIZED</code>), and are displayed when
              appropriate. There are no JSON equivalents for
              <code class="literal">SIMPLE</code> or <code class="literal">PRIMARY</code>.
            </p><p>
              The <code class="literal">select_type</code> value for
              non-<a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements
              displays the statement type for affected tables. For
              example, <code class="literal">select_type</code> is
              <code class="literal">DELETE</code> for
              <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a> statements.
            </p></li><li class="listitem"><p><a name="explain_table"></a>
              <code class="literal">table</code> (JSON name:
              <code class="literal">table_name</code>)
            </p><p>
              The name of the table to which the row of output refers.
              This can also be one of the following values:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                  <code class="literal">&lt;union<em class="replaceable"><code>M</code></em>,<em class="replaceable"><code>N</code></em>&gt;</code>:
                  The row refers to the union of the rows with
                  <code class="literal">id</code> values of
                  <em class="replaceable"><code>M</code></em> and
                  <em class="replaceable"><code>N</code></em>.
                </p></li><li class="listitem"><p>
                  <code class="literal">&lt;derived<em class="replaceable"><code>N</code></em>&gt;</code>:
                  The row refers to the derived table result for the row
                  with an <code class="literal">id</code> value of
                  <em class="replaceable"><code>N</code></em>. A derived table may
                  result, for example, from a subquery in the
                  <code class="literal">FROM</code> clause.
                </p></li><li class="listitem"><p>
                  <code class="literal">&lt;subquery<em class="replaceable"><code>N</code></em>&gt;</code>:
                  The row refers to the result of a materialized
                  subquery for the row with an <code class="literal">id</code>
                  value of <em class="replaceable"><code>N</code></em>. See
                  <a class="xref" href="optimization.html#subquery-materialization" title="8.2.2.2 Optimizing Subqueries with Materialization">Section 8.2.2.2, “Optimizing Subqueries with Materialization”</a>.
</p></li></ul>
</div>
</li><li class="listitem"><p><a name="explain_partitions"></a>
              <code class="literal">partitions</code> (JSON name:
              <code class="literal">partitions</code>)
            </p><p>
              The partitions from which records would be matched by the
              query. The value is <code class="literal">NULL</code> for
              nonpartitioned tables. See
              <a class="xref" href="partitioning.html#partitioning-info" title="23.3.5 Obtaining Information About Partitions">Section 23.3.5, “Obtaining Information About Partitions”</a>.
            </p></li><li class="listitem"><p><a name="explain_type"></a>
              <code class="literal">type</code> (JSON name:
              <code class="literal">access_type</code>)
            </p><p>
              The join type. For descriptions of the different types,
              see
              <a class="link" href="optimization.html#explain-join-types" title="EXPLAIN Join Types"><code class="literal">EXPLAIN</code>
              Join Types</a>.
            </p></li><li class="listitem"><p><a name="explain_possible_keys"></a>
              <code class="literal">possible_keys</code> (JSON name:
              <code class="literal">possible_keys</code>)
            </p><p>
              The <code class="literal">possible_keys</code> column indicates the
              indexes from which MySQL can choose to find the rows in
              this table. Note that this column is totally independent
              of the order of the tables as displayed in the output from
              <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>. That means that
              some of the keys in <code class="literal">possible_keys</code> might
              not be usable in practice with the generated table order.
            </p><p>
              If this column is <code class="literal">NULL</code> (or undefined in
              JSON-formatted output), there are no relevant indexes. In
              this case, you may be able to improve the performance of
              your query by examining the <code class="literal">WHERE</code>
              clause to check whether it refers to some column or
              columns that would be suitable for indexing. If so, create
              an appropriate index and check the query with
              <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> again. See
              <a class="xref" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement">Section 13.1.9, “ALTER TABLE Statement”</a>.
            </p><p>
              To see what indexes a table has, use <code class="literal">SHOW INDEX
              FROM <em class="replaceable"><code>tbl_name</code></em></code>.
            </p></li><li class="listitem"><p><a name="explain_key"></a>
              <code class="literal">key</code> (JSON name: <code class="literal">key</code>)
            </p><p>
              The <code class="literal">key</code> column indicates the key
              (index) that MySQL actually decided to use. If MySQL
              decides to use one of the <code class="literal">possible_keys</code>
              indexes to look up rows, that index is listed as the key
              value.
            </p><p>
              It is possible that <code class="literal">key</code> will name an
              index that is not present in the
              <code class="literal">possible_keys</code> value. This can happen if
              none of the <code class="literal">possible_keys</code> indexes are
              suitable for looking up rows, but all the columns selected
              by the query are columns of some other index. That is, the
              named index covers the selected columns, so although it is
              not used to determine which rows to retrieve, an index
              scan is more efficient than a data row scan.
            </p><p>
              For <code class="literal">InnoDB</code>, a secondary index might
              cover the selected columns even if the query also selects
              the primary key because <code class="literal">InnoDB</code> stores
              the primary key value with each secondary index. If
              <code class="literal">key</code> is <code class="literal">NULL</code>, MySQL
              found no index to use for executing the query more
              efficiently.
            </p><p>
              To force MySQL to use or ignore an index listed in the
              <code class="literal">possible_keys</code> column, use
              <code class="literal">FORCE INDEX</code>, <code class="literal">USE
              INDEX</code>, or <code class="literal">IGNORE INDEX</code> in
              your query. See <a class="xref" href="optimization.html#index-hints" title="8.9.4 Index Hints">Section 8.9.4, “Index Hints”</a>.
            </p><p>
              For <code class="literal">MyISAM</code> tables, running
              <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE TABLE</code></a> helps the
              optimizer choose better indexes. For
              <code class="literal">MyISAM</code> tables, <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk
              --analyze</strong></span></a> does the same. See
              <a class="xref" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement">Section 13.7.3.1, “ANALYZE TABLE Statement”</a>, and
              <a class="xref" href="backup-and-recovery.html#myisam-table-maintenance" title="7.6 MyISAM Table Maintenance and Crash Recovery">Section 7.6, “MyISAM Table Maintenance and Crash Recovery”</a>.
            </p></li><li class="listitem"><p><a name="explain_key_len"></a>
              <code class="literal">key_len</code> (JSON name:
              <code class="literal">key_length</code>)
            </p><p>
              The <code class="literal">key_len</code> column indicates the length
              of the key that MySQL decided to use. The value of
              <code class="literal">key_len</code> enables you to determine how
              many parts of a multiple-part key MySQL actually uses. If
              the <code class="literal">key</code> column says
              <code class="literal">NULL</code>, the <code class="literal">key_len</code>
              column also says <code class="literal">NULL</code>.
            </p><p>
              Due to the key storage format, the key length is one
              greater for a column that can be <code class="literal">NULL</code>
              than for a <code class="literal">NOT NULL</code> column.
            </p></li><li class="listitem"><p><a name="explain_ref"></a>
              <code class="literal">ref</code> (JSON name: <code class="literal">ref</code>)
            </p><p>
              The <code class="literal">ref</code> column shows which columns or
              constants are compared to the index named in the
              <code class="literal">key</code> column to select rows from the
              table.
            </p><p>
              If the value is <code class="literal">func</code>, the value used is
              the result of some function. To see which function, use
              <a class="link" href="sql-statements.html#show-warnings" title="13.7.7.40 SHOW WARNINGS Statement"><code class="literal">SHOW WARNINGS</code></a> following
              <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> to see the extended
              <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output. The
              function might actually be an operator such as an
              arithmetic operator.
            </p></li><li class="listitem"><p><a name="explain_rows"></a>
              <code class="literal">rows</code> (JSON name:
              <code class="literal">rows</code>)
            </p><p>
              The <code class="literal">rows</code> column indicates the number of
              rows MySQL believes it must examine to execute the query.
            </p><p>
              For <a class="link" href="innodb-storage-engine.html" title="Chapter 15 The InnoDB Storage Engine"><code class="literal">InnoDB</code></a> tables, this
              number is an estimate, and may not always be exact.
            </p></li><li class="listitem"><p><a name="explain_filtered"></a>
              <code class="literal">filtered</code> (JSON name:
              <code class="literal">filtered</code>)
            </p><p>
              The <code class="literal">filtered</code> column indicates an
              estimated percentage of table rows that will be filtered
              by the table condition. The maximum value is 100, which
              means no filtering of rows occurred. Values decreasing
              from 100 indicate increasing amounts of filtering.
              <code class="literal">rows</code> shows the estimated number of rows
              examined and <code class="literal">rows</code> ×
              <code class="literal">filtered</code> shows the number of rows that
              will be joined with the following table. For example, if
              <code class="literal">rows</code> is 1000 and
              <code class="literal">filtered</code> is 50.00 (50%), the number of
              rows to be joined with the following table is 1000 ×
              50% = 500.
            </p></li><li class="listitem"><p><a name="explain_extra"></a>
              <code class="literal">Extra</code> (JSON name: none)
            </p><p>
              This column contains additional information about how
              MySQL resolves the query. For descriptions of the
              different values, see
              <a class="link" href="optimization.html#explain-extra-information" title="EXPLAIN Extra Information"><code class="literal">EXPLAIN</code>
              Extra Information</a>.
            </p><p>
              There is no single JSON property corresponding to the
              <code class="literal">Extra</code> column; however, values that can
              occur in this column are exposed as JSON properties, or as
              the text of the <code class="literal">message</code> property.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="explain-join-types"></a>EXPLAIN Join Types</h4>

</div>

</div>

</div>
<p>
          The <code class="literal">type</code> column of
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output describes how
          tables are joined. In JSON-formatted output, these are found
          as values of the <code class="literal">access_type</code> property. The
          following list describes the join types, ordered from the best
          type to the worst:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a name="jointype_system"></a>
              <a class="indexterm" name="idm46444348720496"></a>

              <a class="indexterm" name="idm46444348719040"></a>

              <a class="indexterm" name="idm46444348717552"></a>

              <a class="link" href="optimization.html#jointype_system"><code class="literal">system</code></a>
            </p><p>
              The table has only one row (= system table). This is a
              special case of the
              <a class="link" href="optimization.html#jointype_const"><code class="literal">const</code></a> join type.
            </p></li><li class="listitem"><p><a name="jointype_const"></a>
              <a class="indexterm" name="idm46444348711872"></a>

              <a class="indexterm" name="idm46444348710416"></a>

              <a class="indexterm" name="idm46444348708928"></a>

              <a class="link" href="optimization.html#jointype_const"><code class="literal">const</code></a>
            </p><p>
              The table has at most one matching row, which is read at
              the start of the query. Because there is only one row,
              values from the column in this row can be regarded as
              constants by the rest of the optimizer.
              <a class="link" href="optimization.html#jointype_const"><code class="literal">const</code></a> tables are very
              fast because they are read only once.
            </p><p>
              <a class="link" href="optimization.html#jointype_const"><code class="literal">const</code></a> is used when you
              compare all parts of a <code class="literal">PRIMARY KEY</code> or
              <code class="literal">UNIQUE</code> index to constant values. In the
              following queries, <em class="replaceable"><code>tbl_name</code></em> can
              be used as a <a class="link" href="optimization.html#jointype_const"><code class="literal">const</code></a>
              table:
            </p><pre data-lang="sql" class="programlisting">SELECT * FROM <em class="replaceable"><code>tbl_name</code></em> WHERE <em class="replaceable"><code>primary_key</code></em>=1;

SELECT * FROM <em class="replaceable"><code>tbl_name</code></em>
  WHERE <em class="replaceable"><code>primary_key_part1</code></em>=1 AND <em class="replaceable"><code>primary_key_part2</code></em>=2;
</pre></li><li class="listitem"><p><a name="jointype_eq_ref"></a>
              <a class="indexterm" name="idm46444348695024"></a>

              <a class="indexterm" name="idm46444348693568"></a>

              <a class="link" href="optimization.html#jointype_eq_ref"><code class="literal">eq_ref</code></a>
            </p><p>
              One row is read from this table for each combination of
              rows from the previous tables. Other than the
              <a class="link" href="optimization.html#jointype_system"><code class="literal">system</code></a> and
              <a class="link" href="optimization.html#jointype_const"><code class="literal">const</code></a> types, this is
              the best possible join type. It is used when all parts of
              an index are used by the join and the index is a
              <code class="literal">PRIMARY KEY</code> or <code class="literal">UNIQUE NOT
              NULL</code> index.
            </p><p>
              <a class="link" href="optimization.html#jointype_eq_ref"><code class="literal">eq_ref</code></a> can be used for
              indexed columns that are compared using the
              <code class="literal">=</code> operator. The comparison value can be
              a constant or an expression that uses columns from tables
              that are read before this table. In the following
              examples, MySQL can use an
              <a class="link" href="optimization.html#jointype_eq_ref"><code class="literal">eq_ref</code></a> join to process
              <em class="replaceable"><code>ref_table</code></em>:
            </p><pre data-lang="sql" class="programlisting">SELECT * FROM <em class="replaceable"><code>ref_table</code></em>,<em class="replaceable"><code>other_table</code></em>
  WHERE <em class="replaceable"><code>ref_table</code></em>.<em class="replaceable"><code>key_column</code></em>=<em class="replaceable"><code>other_table</code></em>.<em class="replaceable"><code>column</code></em>;

SELECT * FROM <em class="replaceable"><code>ref_table</code></em>,<em class="replaceable"><code>other_table</code></em>
  WHERE <em class="replaceable"><code>ref_table</code></em>.<em class="replaceable"><code>key_column_part1</code></em>=<em class="replaceable"><code>other_table</code></em>.<em class="replaceable"><code>column</code></em>
  AND <em class="replaceable"><code>ref_table</code></em>.<em class="replaceable"><code>key_column_part2</code></em>=1;
</pre></li><li class="listitem"><p><a name="jointype_ref"></a>
              <a class="indexterm" name="idm46444348673840"></a>

              <a class="indexterm" name="idm46444348672416"></a>

              <a class="link" href="optimization.html#jointype_ref"><code class="literal">ref</code></a>
            </p><p>
              All rows with matching index values are read from this
              table for each combination of rows from the previous
              tables. <a class="link" href="optimization.html#jointype_ref"><code class="literal">ref</code></a> is used if
              the join uses only a leftmost prefix of the key or if the
              key is not a <code class="literal">PRIMARY KEY</code> or
              <code class="literal">UNIQUE</code> index (in other words, if the
              join cannot select a single row based on the key value).
              If the key that is used matches only a few rows, this is a
              good join type.
            </p><p>
              <a class="link" href="optimization.html#jointype_ref"><code class="literal">ref</code></a> can be used for
              indexed columns that are compared using the
              <code class="literal">=</code> or <code class="literal">&lt;=&gt;</code>
              operator. In the following examples, MySQL can use a
              <a class="link" href="optimization.html#jointype_ref"><code class="literal">ref</code></a> join to process
              <em class="replaceable"><code>ref_table</code></em>:
            </p><pre data-lang="sql" class="programlisting">SELECT * FROM <em class="replaceable"><code>ref_table</code></em> WHERE <em class="replaceable"><code>key_column</code></em>=<em class="replaceable"><code>expr</code></em>;

SELECT * FROM <em class="replaceable"><code>ref_table</code></em>,<em class="replaceable"><code>other_table</code></em>
  WHERE <em class="replaceable"><code>ref_table</code></em>.<em class="replaceable"><code>key_column</code></em>=<em class="replaceable"><code>other_table</code></em>.<em class="replaceable"><code>column</code></em>;

SELECT * FROM <em class="replaceable"><code>ref_table</code></em>,<em class="replaceable"><code>other_table</code></em>
  WHERE <em class="replaceable"><code>ref_table</code></em>.<em class="replaceable"><code>key_column_part1</code></em>=<em class="replaceable"><code>other_table</code></em>.<em class="replaceable"><code>column</code></em>
  AND <em class="replaceable"><code>ref_table</code></em>.<em class="replaceable"><code>key_column_part2</code></em>=1;
</pre></li><li class="listitem"><p><a name="jointype_fulltext"></a>
              <a class="indexterm" name="idm46444348651152"></a>

              <a class="indexterm" name="idm46444348649696"></a>

              <a class="link" href="optimization.html#jointype_fulltext"><code class="literal">fulltext</code></a>
            </p><p>
              The join is performed using a <code class="literal">FULLTEXT</code>
              index.
            </p></li><li class="listitem"><p><a name="jointype_ref_or_null"></a>
              <a class="indexterm" name="idm46444348644528"></a>

              <a class="indexterm" name="idm46444348643072"></a>

              <a class="link" href="optimization.html#jointype_ref_or_null"><code class="literal">ref_or_null</code></a>
            </p><p>
              This join type is like
              <a class="link" href="optimization.html#jointype_ref"><code class="literal">ref</code></a>, but with the
              addition that MySQL does an extra search for rows that
              contain <code class="literal">NULL</code> values. This join type
              optimization is used most often in resolving subqueries.
              In the following examples, MySQL can use a
              <a class="link" href="optimization.html#jointype_ref_or_null"><code class="literal">ref_or_null</code></a> join to
              process <em class="replaceable"><code>ref_table</code></em>:
            </p><pre data-lang="sql" class="programlisting">SELECT * FROM <em class="replaceable"><code>ref_table</code></em>
  WHERE <em class="replaceable"><code>key_column</code></em>=<em class="replaceable"><code>expr</code></em> OR <em class="replaceable"><code>key_column</code></em> IS NULL;
</pre><p>
              See <a class="xref" href="optimization.html#is-null-optimization" title="8.2.1.15 IS NULL Optimization">Section 8.2.1.15, “IS NULL Optimization”</a>.
            </p></li><li class="listitem"><p><a name="jointype_index_merge"></a>
              <a class="indexterm" name="idm46444348630944"></a>

              <a class="indexterm" name="idm46444348629488"></a>

              <a class="link" href="optimization.html#jointype_index_merge"><code class="literal">index_merge</code></a>
            </p><p>
              This join type indicates that the Index Merge optimization
              is used. In this case, the <code class="literal">key</code> column
              in the output row contains a list of indexes used, and
              <code class="literal">key_len</code> contains a list of the longest
              key parts for the indexes used. For more information, see
              <a class="xref" href="optimization.html#index-merge-optimization" title="8.2.1.3 Index Merge Optimization">Section 8.2.1.3, “Index Merge Optimization”</a>.
            </p></li><li class="listitem"><p><a name="jointype_unique_subquery"></a>
              <a class="indexterm" name="idm46444348622720"></a>

              <a class="indexterm" name="idm46444348621232"></a>

              <a class="link" href="optimization.html#jointype_unique_subquery"><code class="literal">unique_subquery</code></a>
            </p><p>
              This type replaces
              <a class="link" href="optimization.html#jointype_eq_ref"><code class="literal">eq_ref</code></a> for some
              <code class="literal">IN</code> subqueries of the following form:
            </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>value</code></em> IN (SELECT <em class="replaceable"><code>primary_key</code></em> FROM <em class="replaceable"><code>single_table</code></em> WHERE <em class="replaceable"><code>some_expr</code></em>)
</pre><p>
              <a class="link" href="optimization.html#jointype_unique_subquery"><code class="literal">unique_subquery</code></a> is just
              an index lookup function that replaces the subquery
              completely for better efficiency.
            </p></li><li class="listitem"><p><a name="jointype_index_subquery"></a>
              <a class="indexterm" name="idm46444348610560"></a>

              <a class="indexterm" name="idm46444348609104"></a>

              <a class="link" href="optimization.html#jointype_index_subquery"><code class="literal">index_subquery</code></a>
            </p><p>
              This join type is similar to
              <a class="link" href="optimization.html#jointype_unique_subquery"><code class="literal">unique_subquery</code></a>. It
              replaces <code class="literal">IN</code> subqueries, but it works
              for nonunique indexes in subqueries of the following form:
            </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>value</code></em> IN (SELECT <em class="replaceable"><code>key_column</code></em> FROM <em class="replaceable"><code>single_table</code></em> WHERE <em class="replaceable"><code>some_expr</code></em>)
</pre></li><li class="listitem"><p><a name="jointype_range"></a>
              <a class="indexterm" name="idm46444348600160"></a>

              <a class="indexterm" name="idm46444348598704"></a>

              <a class="link" href="optimization.html#jointype_range"><code class="literal">range</code></a>
            </p><p>
              Only rows that are in a given range are retrieved, using
              an index to select the rows. The <code class="literal">key</code>
              column in the output row indicates which index is used.
              The <code class="literal">key_len</code> contains the longest key
              part that was used. The <code class="literal">ref</code> column is
              <code class="literal">NULL</code> for this type.
            </p><p>
              <a class="link" href="optimization.html#jointype_range"><code class="literal">range</code></a> can be used when
              a key column is compared to a constant using any of the
              <a class="link" href="functions.html#operator_equal"><code class="literal">=</code></a>,
              <a class="link" href="functions.html#operator_not-equal"><code class="literal">&lt;&gt;</code></a>,
              <a class="link" href="functions.html#operator_greater-than"><code class="literal">&gt;</code></a>,
              <a class="link" href="functions.html#operator_greater-than-or-equal"><code class="literal">&gt;=</code></a>,
              <a class="link" href="functions.html#operator_less-than"><code class="literal">&lt;</code></a>,
              <a class="link" href="functions.html#operator_less-than-or-equal"><code class="literal">&lt;=</code></a>,
              <a class="link" href="functions.html#operator_is-null"><code class="literal">IS NULL</code></a>,
              <a class="link" href="functions.html#operator_equal-to"><code class="literal">&lt;=&gt;</code></a>,
              <a class="link" href="functions.html#operator_between"><code class="literal">BETWEEN</code></a>,
              <a class="link" href="functions.html#operator_like"><code class="literal">LIKE</code></a>, or
              <a class="link" href="functions.html#operator_in"><code class="literal">IN()</code></a> operators:
            </p><pre data-lang="sql" class="programlisting">SELECT * FROM <em class="replaceable"><code>tbl_name</code></em>
  WHERE <em class="replaceable"><code>key_column</code></em> = 10;

SELECT * FROM <em class="replaceable"><code>tbl_name</code></em>
  WHERE <em class="replaceable"><code>key_column</code></em> BETWEEN 10 and 20;

SELECT * FROM <em class="replaceable"><code>tbl_name</code></em>
  WHERE <em class="replaceable"><code>key_column</code></em> IN (10,20,30);

SELECT * FROM <em class="replaceable"><code>tbl_name</code></em>
  WHERE <em class="replaceable"><code>key_part1</code></em> = 10 AND <em class="replaceable"><code>key_part2</code></em> IN (10,20,30);
</pre></li><li class="listitem"><p><a name="jointype_index"></a>
              <a class="indexterm" name="idm46444348571008"></a>

              <a class="indexterm" name="idm46444348569552"></a>

              <a class="link" href="optimization.html#jointype_index"><code class="literal">index</code></a>
            </p><p>
              The <code class="literal">index</code> join type is the same as
              <a class="link" href="optimization.html#jointype_all"><code class="literal">ALL</code></a>, except that the
              index tree is scanned. This occurs two ways:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                  If the index is a covering index for the queries and
                  can be used to satisfy all data required from the
                  table, only the index tree is scanned. In this case,
                  the <code class="literal">Extra</code> column says
                  <code class="literal">Using index</code>. An index-only scan
                  usually is faster than
                  <a class="link" href="optimization.html#jointype_all"><code class="literal">ALL</code></a> because the
                  size of the index usually is smaller than the table
                  data.
                </p></li><li class="listitem"><p>
                  A full table scan is performed using reads from the
                  index to look up data rows in index order.
                  <code class="literal">Uses index</code> does not appear in the
                  <code class="literal">Extra</code> column.
</p></li></ul>
</div>
<p>
              MySQL can use this join type when the query uses only
              columns that are part of a single index.
            </p></li><li class="listitem"><p><a name="jointype_all"></a>
              <a class="indexterm" name="idm46444348555984"></a>

              <a class="indexterm" name="idm46444348554560"></a>

              <a class="link" href="optimization.html#jointype_all"><code class="literal">ALL</code></a>
            </p><p>
              A full table scan is done for each combination of rows
              from the previous tables. This is normally not good if the
              table is the first table not marked
              <a class="link" href="optimization.html#jointype_const"><code class="literal">const</code></a>, and usually
              <span class="emphasis"><em>very</em></span> bad in all other cases.
              Normally, you can avoid
              <a class="link" href="optimization.html#jointype_all"><code class="literal">ALL</code></a> by adding indexes
              that enable row retrieval from the table based on constant
              values or column values from earlier tables.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="explain-extra-information"></a>EXPLAIN Extra Information</h4>

</div>

</div>

</div>
<p>
          The <code class="literal">Extra</code> column of
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output contains
          additional information about how MySQL resolves the query. The
          following list explains the values that can appear in this
          column. Each item also indicates for JSON-formatted output
          which property displays the <code class="literal">Extra</code> value.
          For some of these, there is a specific property. The others
          display as the text of the <code class="literal">message</code>
          property.
        </p><p>
          If you want to make your queries as fast as possible, look out
          for <code class="literal">Extra</code> column values of <code class="literal">Using
          filesort</code> and <code class="literal">Using temporary</code>, or,
          in JSON-formatted <code class="literal">EXPLAIN</code> output, for
          <code class="literal">using_filesort</code> and
          <code class="literal">using_temporary_table</code> properties equal to
          <code class="literal">true</code>.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <code class="literal">Child of '<em class="replaceable"><code>table</code></em>'
              pushed join@1</code> (JSON: <code class="literal">message</code>
              text)
            </p><p>
              This table is referenced as the child of
              <em class="replaceable"><code>table</code></em> in a join that can be
              pushed down to the NDB kernel. Applies only in NDB
              Cluster, when pushed-down joins are enabled. See the
              description of the
              <a class="link" href="mysql-cluster.html#sysvar_ndb_join_pushdown"><code class="literal">ndb_join_pushdown</code></a> server
              system variable for more information and examples.
            </p></li><li class="listitem"><p>
              <code class="literal">const row not found</code> (JSON property:
              <code class="literal">const_row_not_found</code>)
            </p><p>
              For a query such as <code class="literal">SELECT ... FROM
              <em class="replaceable"><code>tbl_name</code></em></code>, the table
              was empty.
            </p></li><li class="listitem"><p>
              <code class="literal">Deleting all rows</code> (JSON property:
              <code class="literal">message</code>)
            </p><p>
              For <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a>, some storage
              engines (such as <a class="link" href="storage-engines.html#myisam-storage-engine" title="16.2 The MyISAM Storage Engine"><code class="literal">MyISAM</code></a>)
              support a handler method that removes all table rows in a
              simple and fast way. This <code class="literal">Extra</code> value
              is displayed if the engine uses this optimization.
            </p></li><li class="listitem"><p>
              <code class="literal">Distinct</code> (JSON property:
              <code class="literal">distinct</code>)
            </p><p>
              MySQL is looking for distinct values, so it stops
              searching for more rows for the current row combination
              after it has found the first matching row.
            </p></li><li class="listitem"><p>
              <code class="literal">FirstMatch(<em class="replaceable"><code>tbl_name</code></em>)</code>
              (JSON property: <code class="literal">first_match</code>)
            </p><p>
              The semijoin FirstMatch join shortcutting strategy is used
              for <em class="replaceable"><code>tbl_name</code></em>.
            </p></li><li class="listitem"><p>
              <code class="literal">Full scan on NULL key</code> (JSON property:
              <code class="literal">message</code>)
            </p><p>
              This occurs for subquery optimization as a fallback
              strategy when the optimizer cannot use an index-lookup
              access method.
            </p></li><li class="listitem"><p>
              <code class="literal">Impossible HAVING</code> (JSON property:
              <code class="literal">message</code>)
            </p><p>
              The <code class="literal">HAVING</code> clause is always false and
              cannot select any rows.
            </p></li><li class="listitem"><p>
              <code class="literal">Impossible WHERE</code> (JSON property:
              <code class="literal">message</code>)
            </p><p>
              The <code class="literal">WHERE</code> clause is always false and
              cannot select any rows.
            </p></li><li class="listitem"><p>
              <code class="literal">Impossible WHERE noticed after reading const
              tables</code> (JSON property:
              <code class="literal">message</code>)
            </p><p>
              MySQL has read all
              <a class="link" href="optimization.html#jointype_const"><code class="literal">const</code></a> (and
              <a class="link" href="optimization.html#jointype_system"><code class="literal">system</code></a>) tables and
              notice that the <code class="literal">WHERE</code> clause is always
              false.
            </p></li><li class="listitem"><p>
              <code class="literal">LooseScan(<em class="replaceable"><code>m</code></em>..<em class="replaceable"><code>n</code></em>)</code>
              (JSON property: <code class="literal">message</code>)
            </p><p>
              The semijoin LooseScan strategy is used.
              <em class="replaceable"><code>m</code></em> and
              <em class="replaceable"><code>n</code></em> are key part numbers.
            </p></li><li class="listitem"><p>
              <code class="literal">No matching min/max row</code> (JSON property:
              <code class="literal">message</code>)
            </p><p>
              No row satisfies the condition for a query such as
              <code class="literal">SELECT MIN(...) FROM ... WHERE
              <em class="replaceable"><code>condition</code></em></code>.
            </p></li><li class="listitem"><p>
              <code class="literal">no matching row in const table</code> (JSON
              property: <code class="literal">message</code>)
            </p><p>
              For a query with a join, there was an empty table or a
              table with no rows satisfying a unique index condition.
            </p></li><li class="listitem"><p>
              <code class="literal">No matching rows after partition
              pruning</code> (JSON property:
              <code class="literal">message</code>)
            </p><p>
              For <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a> or
              <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a>, the optimizer found
              nothing to delete or update after partition pruning. It is
              similar in meaning to <code class="literal">Impossible WHERE</code>
              for <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements.
            </p></li><li class="listitem"><p>
              <code class="literal">No tables used</code> (JSON property:
              <code class="literal">message</code>)
            </p><p>
              The query has no <code class="literal">FROM</code> clause, or has a
              <code class="literal">FROM DUAL</code> clause.
            </p><p>
              For <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a> or
              <a class="link" href="sql-statements.html#replace" title="13.2.9 REPLACE Statement"><code class="literal">REPLACE</code></a> statements,
              <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> displays this value
              when there is no <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>
              part. For example, it appears for <code class="literal">EXPLAIN INSERT
              INTO t VALUES(10)</code> because that is equivalent to
              <code class="literal">EXPLAIN INSERT INTO t SELECT 10 FROM
              DUAL</code>.
            </p></li><li class="listitem"><p>
              <code class="literal">Not exists</code> (JSON property:
              <code class="literal">message</code>)
            </p><p>
              MySQL was able to do a <code class="literal">LEFT JOIN</code>
              optimization on the query and does not examine more rows
              in this table for the previous row combination after it
              finds one row that matches the <code class="literal">LEFT
              JOIN</code> criteria. Here is an example of the type of
              query that can be optimized this way:
            </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 LEFT JOIN t2 ON t1.id=t2.id
  WHERE t2.id IS NULL;</pre><p>
              Assume that <code class="literal">t2.id</code> is defined as
              <code class="literal">NOT NULL</code>. In this case, MySQL scans
              <code class="literal">t1</code> and looks up the rows in
              <code class="literal">t2</code> using the values of
              <code class="literal">t1.id</code>. If MySQL finds a matching row in
              <code class="literal">t2</code>, it knows that
              <code class="literal">t2.id</code> can never be
              <code class="literal">NULL</code>, and does not scan through the
              rest of the rows in <code class="literal">t2</code> that have the
              same <code class="literal">id</code> value. In other words, for each
              row in <code class="literal">t1</code>, MySQL needs to do only a
              single lookup in <code class="literal">t2</code>, regardless of how
              many rows actually match in <code class="literal">t2</code>.
            </p><p>
              In MySQL 8.0.17 and later, this can also indicate that a
              <code class="literal">WHERE</code> condition of the form
              <code class="literal">NOT IN
              (<em class="replaceable"><code>subquery</code></em>)</code> or
              <code class="literal">NOT EXISTS
              (<em class="replaceable"><code>subquery</code></em>)</code> has been
              transformed internally into an antijoin. This removes the
              subquery and brings its tables into the plan for the
              topmost query, providing improved cost planning. By
              merging semijoins and antijoins, the optimizer can reorder
              tables in the execution plan more freely, in some cases
              resulting in a faster plan.
            </p><p>
              You can see when an antijoin transformation is performed
              for a given query by checking the
              <code class="literal">Message</code> column from <code class="literal">SHOW
              WARNINGS</code> following execution of
              <code class="literal">EXPLAIN</code>, or in the output of
              <code class="literal">EXPLAIN FORMAT=TREE</code>.
</p>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
                An antijoin is the complement of a semijoin
                <code class="literal"><em class="replaceable"><code>table_a</code></em> JOIN
                <em class="replaceable"><code>table_b</code></em> ON
                <em class="replaceable"><code>condition</code></em></code>. The
                antijoin returns all rows from
                <em class="replaceable"><code>table_a</code></em> for which there is
                <span class="emphasis"><em>no</em></span> row in
                <em class="replaceable"><code>table_b</code></em> which matches
                <em class="replaceable"><code>condition</code></em>.
</p>
</div>
</li><li class="listitem"><p>
              <code class="literal">Plan isn't ready yet</code> (JSON property:
              none)
            </p><p>
              This value occurs with <a class="link" href="optimization.html#explain-for-connection" title="8.8.4 Obtaining Execution Plan Information for a Named Connection"><code class="literal">EXPLAIN FOR
              CONNECTION</code></a> when the optimizer has not finished
              creating the execution plan for the statement executing in
              the named connection. If execution plan output comprises
              multiple lines, any or all of them could have this
              <code class="literal">Extra</code> value, depending on the progress
              of the optimizer in determining the full execution plan.
            </p></li><li class="listitem"><p>
              <code class="literal">Range checked for each record (index map:
              <em class="replaceable"><code>N</code></em>)</code> (JSON property:
              <code class="literal">message</code>)
            </p><p>
              MySQL found no good index to use, but found that some of
              indexes might be used after column values from preceding
              tables are known. For each row combination in the
              preceding tables, MySQL checks whether it is possible to
              use a <a class="link" href="optimization.html#jointype_range"><code class="literal">range</code></a> or
              <a class="link" href="optimization.html#jointype_index_merge"><code class="literal">index_merge</code></a> access
              method to retrieve rows. This is not very fast, but is
              faster than performing a join with no index at all. The
              applicability criteria are as described in
              <a class="xref" href="optimization.html#range-optimization" title="8.2.1.2 Range Optimization">Section 8.2.1.2, “Range Optimization”</a>, and
              <a class="xref" href="optimization.html#index-merge-optimization" title="8.2.1.3 Index Merge Optimization">Section 8.2.1.3, “Index Merge Optimization”</a>, with the
              exception that all column values for the preceding table
              are known and considered to be constants.
            </p><p>
              Indexes are numbered beginning with 1, in the same order
              as shown by <a class="link" href="sql-statements.html#show-index" title="13.7.7.22 SHOW INDEX Statement"><code class="literal">SHOW INDEX</code></a> for
              the table. The index map value
              <em class="replaceable"><code>N</code></em> is a bitmask value that
              indicates which indexes are candidates. For example, a
              value of <code class="literal">0x19</code> (binary 11001) means that
              indexes 1, 4, and 5 will be considered.
            </p></li><li class="listitem"><p>
              <code class="literal">Recursive</code> (JSON property:
              <code class="literal">recursive</code>)
            </p><p>
              This indicates that the row applies to the recursive
              <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> part of a recursive
              common table expression. See <a class="xref" href="sql-statements.html#with" title="13.2.15 WITH (Common Table Expressions)">Section 13.2.15, “WITH (Common Table Expressions)”</a>.
            </p></li><li class="listitem"><p>
              <code class="literal">Rematerialize</code> (JSON property:
              <code class="literal">rematerialize</code>)
            </p><a class="indexterm" name="idm46444348421680"></a><a class="indexterm" name="idm46444348420192"></a><p>
              <code class="literal">Rematerialize (X,...)</code> is displayed in
              the <code class="literal">EXPLAIN</code> row for table
              <code class="literal">T</code>, where <code class="literal">X</code> is any
              lateral derived table whose rematerialization is triggered
              when a new row of <code class="literal">T</code> is read. For
              example:
            </p><pre data-lang="sql" class="programlisting">SELECT
  ...
FROM
  t,
  LATERAL (<em class="replaceable"><code>derived table that refers to t</code></em>) AS dt
...
</pre><p>
              The content of the derived table is rematerialized to
              bring it up to date each time a new row of
              <code class="literal">t</code> is processed by the top query.
            </p></li><li class="listitem"><p>
              <code class="literal">Scanned <em class="replaceable"><code>N</code></em>
              databases</code> (JSON property:
              <code class="literal">message</code>)
            </p><p>
              This indicates how many directory scans the server
              performs when processing a query for
              <code class="literal">INFORMATION_SCHEMA</code> tables, as described
              in <a class="xref" href="optimization.html#information-schema-optimization" title="8.2.3 Optimizing INFORMATION_SCHEMA Queries">Section 8.2.3, “Optimizing INFORMATION_SCHEMA Queries”</a>. The
              value of <em class="replaceable"><code>N</code></em> can be 0, 1, or
              <code class="literal">all</code>.
            </p></li><li class="listitem"><p>
              <code class="literal">Select tables optimized away</code> (JSON
              property: <code class="literal">message</code>)
            </p><p>
              The optimizer determined 1) that at most one row should be
              returned, and 2) that to produce this row, a deterministic
              set of rows must be read. When the rows to be read can be
              read during the optimization phase (for example, by
              reading index rows), there is no need to read any tables
              during query execution.
            </p><p>
              The first condition is fulfilled when the query is
              implicitly grouped (contains an aggregate function but no
              <code class="literal">GROUP BY</code> clause). The second condition
              is fulfilled when one row lookup is performed per index
              used. The number of indexes read determines the number of
              rows to read.
            </p><p>
              Consider the following implicitly grouped query:
            </p><pre data-lang="sql" class="programlisting">SELECT MIN(c1), MIN(c2) FROM t1;</pre><p>
              Suppose that <code class="literal">MIN(c1)</code> can be retrieved
              by reading one index row and <code class="literal">MIN(c2)</code>
              can be retrieved by reading one row from a different
              index. That is, for each column <code class="literal">c1</code> and
              <code class="literal">c2</code>, there exists an index where the
              column is the first column of the index. In this case, one
              row is returned, produced by reading two deterministic
              rows.
            </p><p>
              This <code class="literal">Extra</code> value does not occur if the
              rows to read are not deterministic. Consider this query:
            </p><pre data-lang="sql" class="programlisting">SELECT MIN(c2) FROM t1 WHERE c1 &lt;= 10;</pre><p>
              Suppose that <code class="literal">(c1, c2)</code> is a covering
              index. Using this index, all rows with <code class="literal">c1 &lt;=
              10</code> must be scanned to find the minimum
              <code class="literal">c2</code> value. By contrast, consider this
              query:
            </p><pre data-lang="sql" class="programlisting">SELECT MIN(c2) FROM t1 WHERE c1 = 10;</pre><p>
              In this case, the first index row with <code class="literal">c1 =
              10</code> contains the minimum <code class="literal">c2</code>
              value. Only one row must be read to produce the returned
              row.
            </p><p>
              For storage engines that maintain an exact row count per
              table (such as <code class="literal">MyISAM</code>, but not
              <code class="literal">InnoDB</code>), this <code class="literal">Extra</code>
              value can occur for <code class="literal">COUNT(*)</code> queries
              for which the <code class="literal">WHERE</code> clause is missing
              or always true and there is no <code class="literal">GROUP BY</code>
              clause. (This is an instance of an implicitly grouped
              query where the storage engine influences whether a
              deterministic number of rows can be read.)
            </p></li><li class="listitem"><p>
              <code class="literal">Skip_open_table</code>,
              <code class="literal">Open_frm_only</code>,
              <code class="literal">Open_full_table</code> (JSON property:
              <code class="literal">message</code>)
            </p><p>
              These values indicate file-opening optimizations that
              apply to queries for <code class="literal">INFORMATION_SCHEMA</code>
              tables.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                  <code class="literal">Skip_open_table</code>: Table files do not
                  need to be opened. The information is already
                  available from the data dictionary.
                </p></li><li class="listitem"><p>
                  <code class="literal">Open_frm_only</code>: Only the data
                  dictionary need be read for table information.
                </p></li><li class="listitem"><p>
                  <code class="literal">Open_full_table</code>: Unoptimized
                  information lookup. Table information must be read
                  from the data dictionary and by reading table files.
</p></li></ul>
</div>
</li><li class="listitem"><p>
              <code class="literal">Start temporary</code>, <code class="literal">End
              temporary</code> (JSON property:
              <code class="literal">message</code>)
            </p><p>
              This indicates temporary table use for the semijoin
              Duplicate Weedout strategy.
            </p></li><li class="listitem"><p>
              <code class="literal">unique row not found</code> (JSON property:
              <code class="literal">message</code>)
            </p><p>
              For a query such as <code class="literal">SELECT ... FROM
              <em class="replaceable"><code>tbl_name</code></em></code>, no rows
              satisfy the condition for a <code class="literal">UNIQUE</code>
              index or <code class="literal">PRIMARY KEY</code> on the table.
            </p></li><li class="listitem"><p>
              <code class="literal">Using filesort</code> (JSON property:
              <code class="literal">using_filesort</code>)
            </p><p>
              MySQL must do an extra pass to find out how to retrieve
              the rows in sorted order. The sort is done by going
              through all rows according to the join type and storing
              the sort key and pointer to the row for all rows that
              match the <code class="literal">WHERE</code> clause. The keys then
              are sorted and the rows are retrieved in sorted order. See
              <a class="xref" href="optimization.html#order-by-optimization" title="8.2.1.16 ORDER BY Optimization">Section 8.2.1.16, “ORDER BY Optimization”</a>.
            </p></li><li class="listitem"><p>
              <code class="literal">Using index</code> (JSON property:
              <code class="literal">using_index</code>)
            </p><p>
              The column information is retrieved from the table using
              only information in the index tree without having to do an
              additional seek to read the actual row. This strategy can
              be used when the query uses only columns that are part of
              a single index.
            </p><p>
              For <code class="literal">InnoDB</code> tables that have a
              user-defined clustered index, that index can be used even
              when <code class="literal">Using index</code> is absent from the
              <code class="literal">Extra</code> column. This is the case if
              <code class="literal">type</code> is
              <a class="link" href="optimization.html#jointype_index"><code class="literal">index</code></a> and
              <code class="literal">key</code> is <code class="literal">PRIMARY</code>.
            </p></li><li class="listitem"><p>
              <code class="literal">Using index condition</code> (JSON property:
              <code class="literal">using_index_condition</code>)
            </p><p>
              Tables are read by accessing index tuples and testing them
              first to determine whether to read full table rows. In
              this way, index information is used to defer (<span class="quote">“<span class="quote">push
              down</span>”</span>) reading full table rows unless it is
              necessary. See
              <a class="xref" href="optimization.html#index-condition-pushdown-optimization" title="8.2.1.6 Index Condition Pushdown Optimization">Section 8.2.1.6, “Index Condition Pushdown Optimization”</a>.
            </p></li><li class="listitem"><p>
              <code class="literal">Using index for group-by</code> (JSON
              property: <code class="literal">using_index_for_group_by</code>)
            </p><p>
              Similar to the <code class="literal">Using index</code> table access
              method, <code class="literal">Using index for group-by</code>
              indicates that MySQL found an index that can be used to
              retrieve all columns of a <code class="literal">GROUP BY</code> or
              <code class="literal">DISTINCT</code> query without any extra disk
              access to the actual table. Additionally, the index is
              used in the most efficient way so that for each group,
              only a few index entries are read. For details, see
              <a class="xref" href="optimization.html#group-by-optimization" title="8.2.1.17 GROUP BY Optimization">Section 8.2.1.17, “GROUP BY Optimization”</a>.
            </p></li><li class="listitem"><p>
              <code class="literal">Using index for skip scan</code> (JSON
              property: <code class="literal">using_index_for_skip_scan</code>)
            </p><p>
              Indicates that the Skip Scan access method is used. See
              <a class="xref" href="optimization.html#range-access-skip-scan" title="Skip Scan Range Access Method">Skip Scan Range Access Method</a>.
            </p></li><li class="listitem"><p>
              <code class="literal">Using join buffer (Block Nested Loop)</code>,
              <code class="literal">Using join buffer (Batched Key Access)</code>
              (JSON property: <code class="literal">using_join_buffer</code>)
            </p><p>
              Tables from earlier joins are read in portions into the
              join buffer, and then their rows are used from the buffer
              to perform the join with the current table.
              <code class="literal">(Block Nested Loop)</code> indicates use of
              the Block Nested-Loop algorithm and <code class="literal">(Batched Key
              Access)</code> indicates use of the Batched Key Access
              algorithm. That is, the keys from the table on the
              preceding line of the
              <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output will be
              buffered, and the matching rows will be fetched in batches
              from the table represented by the line in which
              <code class="literal">Using join buffer</code> appears.
            </p><p>
              In JSON-formatted output, the value of
              <code class="literal">using_join_buffer</code> is always either one
              of <code class="literal">Block Nested Loop</code> or
              <code class="literal">Batched Key Access</code>.
            </p></li><li class="listitem"><p>
              <code class="literal">Using MRR</code> (JSON property:
              <code class="literal">message</code>)
            </p><p>
              Tables are read using the Multi-Range Read optimization
              strategy. See <a class="xref" href="optimization.html#mrr-optimization" title="8.2.1.11 Multi-Range Read Optimization">Section 8.2.1.11, “Multi-Range Read Optimization”</a>.
            </p></li><li class="listitem"><p>
              <code class="literal">Using sort_union(...)</code>, <code class="literal">Using
              union(...)</code>, <code class="literal">Using
              intersect(...)</code> (JSON property:
              <code class="literal">message</code>)
            </p><p>
              These indicate the particular algorithm showing how index
              scans are merged for the
              <a class="link" href="optimization.html#jointype_index_merge"><code class="literal">index_merge</code></a> join type.
              See <a class="xref" href="optimization.html#index-merge-optimization" title="8.2.1.3 Index Merge Optimization">Section 8.2.1.3, “Index Merge Optimization”</a>.
            </p></li><li class="listitem"><p>
              <code class="literal">Using temporary</code> (JSON property:
              <code class="literal">using_temporary_table</code>)
            </p><p>
              To resolve the query, MySQL needs to create a temporary
              table to hold the result. This typically happens if the
              query contains <code class="literal">GROUP BY</code> and
              <code class="literal">ORDER BY</code> clauses that list columns
              differently.
            </p></li><li class="listitem"><p>
              <code class="literal">Using where</code> (JSON property:
              <code class="literal">attached_condition</code>)
            </p><p>
              A <code class="literal">WHERE</code> clause is used to restrict
              which rows to match against the next table or send to the
              client. Unless you specifically intend to fetch or examine
              all rows from the table, you may have something wrong in
              your query if the <code class="literal">Extra</code> value is not
              <code class="literal">Using where</code> and the table join type is
              <a class="link" href="optimization.html#jointype_all"><code class="literal">ALL</code></a> or
              <a class="link" href="optimization.html#jointype_index"><code class="literal">index</code></a>.
            </p><p>
              <code class="literal">Using where</code> has no direct counterpart
              in JSON-formatted output; the
              <code class="literal">attached_condition</code> property contains
              any <code class="literal">WHERE</code> condition used.
            </p></li><li class="listitem"><p>
              <code class="literal">Using where with pushed condition</code> (JSON
              property: <code class="literal">message</code>)
            </p><p>
              This item applies to <a class="link" href="mysql-cluster.html" title="Chapter 22 MySQL NDB Cluster 8.0"><code class="literal">NDB</code></a>
              tables <span class="emphasis"><em>only</em></span>. It means that NDB
              Cluster is using the Condition Pushdown optimization to
              improve the efficiency of a direct comparison between a
              nonindexed column and a constant. In such cases, the
              condition is <span class="quote">“<span class="quote">pushed down</span>”</span> to the
              cluster's data nodes and is evaluated on all data
              nodes simultaneously. This eliminates the need to send
              nonmatching rows over the network, and can speed up such
              queries by a factor of 5 to 10 times over cases where
              Condition Pushdown could be but is not used. For more
              information, see
              <a class="xref" href="optimization.html#condition-pushdown-optimization" title="8.2.1.5 Engine Condition Pushdown Optimization">Section 8.2.1.5, “Engine Condition Pushdown Optimization”</a>.
            </p></li><li class="listitem"><p>
              <code class="literal">Zero limit</code> (JSON property:
              <code class="literal">message</code>)
            </p><p>
              The query had a <code class="literal">LIMIT 0</code> clause and
              cannot select any rows.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="explain-output-interpretation"></a>EXPLAIN Output Interpretation</h4>

</div>

</div>

</div>
<p>
          You can get a good indication of how good a join is by taking
          the product of the values in the <code class="literal">rows</code>
          column of the <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output.
          This should tell you roughly how many rows MySQL must examine
          to execute the query. If you restrict queries with the
          <a class="link" href="server-administration.html#sysvar_max_join_size"><code class="literal">max_join_size</code></a> system
          variable, this row product also is used to determine which
          multiple-table <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>
          statements to execute and which to abort. See
          <a class="xref" href="server-administration.html#server-configuration" title="5.1.1 Configuring the Server">Section 5.1.1, “Configuring the Server”</a>.
        </p><p>
          The following example shows how a multiple-table join can be
          optimized progressively based on the information provided by
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>.
        </p><p>
          Suppose that you have the
          <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statement shown here and
          that you plan to examine it using
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>:
        </p><pre data-lang="sql" class="programlisting">EXPLAIN SELECT tt.TicketNumber, tt.TimeIn,
               tt.ProjectReference, tt.EstimatedShipDate,
               tt.ActualShipDate, tt.ClientID,
               tt.ServiceCodes, tt.RepetitiveID,
               tt.CurrentProcess, tt.CurrentDPPerson,
               tt.RecordVolume, tt.DPPrinted, et.COUNTRY,
               et_1.COUNTRY, do.CUSTNAME
        FROM tt, et, et AS et_1, do
        WHERE tt.SubmitTime IS NULL
          AND tt.ActualPC = et.EMPLOYID
          AND tt.AssignedPC = et_1.EMPLOYID
          AND tt.ClientID = do.CUSTNMBR;</pre><p>
          For this example, make the following assumptions:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              The columns being compared have been declared as follows.
</p>
<div class="informaltable">
<table summary="Table names, column names, and data types for the columns being compared in the EXPLAIN example described in the preceding text."><col width="10%"><col width="25%"><col width="25%"><thead><tr>
                  <th scope="col">Table</th>
                  <th scope="col">Column</th>
                  <th scope="col">Data Type</th>
                </tr></thead><tbody><tr>
                  <td scope="row"><code class="literal">tt</code></td>
                  <td><code class="literal">ActualPC</code></td>
                  <td><code class="literal">CHAR(10)</code></td>
                </tr><tr>
                  <td scope="row"><code class="literal">tt</code></td>
                  <td><code class="literal">AssignedPC</code></td>
                  <td><code class="literal">CHAR(10)</code></td>
                </tr><tr>
                  <td scope="row"><code class="literal">tt</code></td>
                  <td><code class="literal">ClientID</code></td>
                  <td><code class="literal">CHAR(10)</code></td>
                </tr><tr>
                  <td scope="row"><code class="literal">et</code></td>
                  <td><code class="literal">EMPLOYID</code></td>
                  <td><code class="literal">CHAR(15)</code></td>
                </tr><tr>
                  <td scope="row"><code class="literal">do</code></td>
                  <td><code class="literal">CUSTNMBR</code></td>
                  <td><code class="literal">CHAR(15)</code></td>
</tr></tbody></table>
</div>
</li><li class="listitem"><p>
              The tables have the following indexes.
</p>
<div class="informaltable">
<table summary="Indexes for each of the tables that are part of the EXPLAIN example described in the preceding text."><col width="10%"><col width="40%"><thead><tr>
                  <th scope="col">Table</th>
                  <th scope="col">Index</th>
                </tr></thead><tbody><tr>
                  <td scope="row"><code class="literal">tt</code></td>
                  <td><code class="literal">ActualPC</code></td>
                </tr><tr>
                  <td scope="row"><code class="literal">tt</code></td>
                  <td><code class="literal">AssignedPC</code></td>
                </tr><tr>
                  <td scope="row"><code class="literal">tt</code></td>
                  <td><code class="literal">ClientID</code></td>
                </tr><tr>
                  <td scope="row"><code class="literal">et</code></td>
                  <td><code class="literal">EMPLOYID</code> (primary key)</td>
                </tr><tr>
                  <td scope="row"><code class="literal">do</code></td>
                  <td><code class="literal">CUSTNMBR</code> (primary key)</td>
</tr></tbody></table>
</div>
</li><li class="listitem"><p>
              The <code class="literal">tt.ActualPC</code> values are not evenly
              distributed.
</p></li></ul>
</div>
<p>
          Initially, before any optimizations have been performed, the
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> statement produces the
          following information:
        </p><pre data-lang="none" class="programlisting">table type possible_keys key  key_len ref  rows  Extra
et    ALL  PRIMARY       NULL NULL    NULL 74
do    ALL  PRIMARY       NULL NULL    NULL 2135
et_1  ALL  PRIMARY       NULL NULL    NULL 74
tt    ALL  AssignedPC,   NULL NULL    NULL 3872
           ClientID,
           ActualPC
      Range checked for each record (index map: 0x23)</pre><p>
          Because <code class="literal">type</code> is
          <a class="link" href="optimization.html#jointype_all"><code class="literal">ALL</code></a> for each table, this
          output indicates that MySQL is generating a Cartesian product
          of all the tables; that is, every combination of rows. This
          takes quite a long time, because the product of the number of
          rows in each table must be examined. For the case at hand,
          this product is 74 × 2135 × 74 × 3872 =
          45,268,558,720 rows. If the tables were bigger, you can only
          imagine how long it would take.
        </p><p>
          One problem here is that MySQL can use indexes on columns more
          efficiently if they are declared as the same type and size. In
          this context, <a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">VARCHAR</code></a> and
          <a class="link" href="data-types.html#char" title="11.3.2 The CHAR and VARCHAR Types"><code class="literal">CHAR</code></a> are considered the same if
          they are declared as the same size.
          <code class="literal">tt.ActualPC</code> is declared as
          <code class="literal">CHAR(10)</code> and <code class="literal">et.EMPLOYID</code>
          is <code class="literal">CHAR(15)</code>, so there is a length mismatch.
        </p><p>
          To fix this disparity between column lengths, use
          <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER TABLE</code></a> to lengthen
          <code class="literal">ActualPC</code> from 10 characters to 15
          characters:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>ALTER TABLE tt MODIFY ActualPC VARCHAR(15);</code></strong>
</pre><p>
          Now <code class="literal">tt.ActualPC</code> and
          <code class="literal">et.EMPLOYID</code> are both
          <code class="literal">VARCHAR(15)</code>. Executing the
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> statement again
          produces this result:
        </p><pre data-lang="none" class="programlisting">table type   possible_keys key     key_len ref         rows    Extra
tt    ALL    AssignedPC,   NULL    NULL    NULL        3872    Using
             ClientID,                                         where
             ActualPC
do    ALL    PRIMARY       NULL    NULL    NULL        2135
      Range checked for each record (index map: 0x1)
et_1  ALL    PRIMARY       NULL    NULL    NULL        74
      Range checked for each record (index map: 0x1)
et    eq_ref PRIMARY       PRIMARY 15      tt.ActualPC 1</pre><p>
          This is not perfect, but is much better: The product of the
          <code class="literal">rows</code> values is less by a factor of 74. This
          version executes in a couple of seconds.
        </p><p>
          A second alteration can be made to eliminate the column length
          mismatches for the <code class="literal">tt.AssignedPC =
          et_1.EMPLOYID</code> and <code class="literal">tt.ClientID =
          do.CUSTNMBR</code> comparisons:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>ALTER TABLE tt MODIFY AssignedPC VARCHAR(15),</code></strong>
                      <strong class="userinput"><code>MODIFY ClientID   VARCHAR(15);</code></strong>
</pre><p>
          After that modification,
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> produces the output
          shown here:
        </p><pre data-lang="none" class="programlisting">table type   possible_keys key      key_len ref           rows Extra
et    ALL    PRIMARY       NULL     NULL    NULL          74
tt    ref    AssignedPC,   ActualPC 15      et.EMPLOYID   52   Using
             ClientID,                                         where
             ActualPC
et_1  eq_ref PRIMARY       PRIMARY  15      tt.AssignedPC 1
do    eq_ref PRIMARY       PRIMARY  15      tt.ClientID   1</pre><p>
          At this point, the query is optimized almost as well as
          possible. The remaining problem is that, by default, MySQL
          assumes that values in the <code class="literal">tt.ActualPC</code>
          column are evenly distributed, and that is not the case for
          the <code class="literal">tt</code> table. Fortunately, it is easy to
          tell MySQL to analyze the key distribution:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>ANALYZE TABLE tt;</code></strong>
</pre><p>
          With the additional index information, the join is perfect and
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> produces this result:
        </p><pre data-lang="none" class="programlisting">table type   possible_keys key     key_len ref           rows Extra
tt    ALL    AssignedPC    NULL    NULL    NULL          3872 Using
             ClientID,                                        where
             ActualPC
et    eq_ref PRIMARY       PRIMARY 15      tt.ActualPC   1
et_1  eq_ref PRIMARY       PRIMARY 15      tt.AssignedPC 1
do    eq_ref PRIMARY       PRIMARY 15      tt.ClientID   1</pre><p>
          <a class="indexterm" name="idm46444348187024"></a>

          The <code class="literal">rows</code> column in the output from
          <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> is an educated guess
          from the MySQL join optimizer. Check whether the numbers are
          even close to the truth by comparing the
          <code class="literal">rows</code> product with the actual number of rows
          that the query returns. If the numbers are quite different,
          you might get better performance by using
          <code class="literal">STRAIGHT_JOIN</code> in your
          <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statement and trying to
          list the tables in a different order in the
          <code class="literal">FROM</code> clause. (However,
          <code class="literal">STRAIGHT_JOIN</code> may prevent indexes from
          being used because it disables semijoin transformations. See
          <a class="xref" href="optimization.html#semijoins" title="8.2.2.1 Optimizing IN and EXISTS Subquery Predicates with Semijoin Transformations">Section 8.2.2.1, “Optimizing IN and EXISTS Subquery Predicates with Semijoin
          Transformations”</a>.)
        </p><p>
          It is possible in some cases to execute statements that modify
          data when <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN
          SELECT</code></a> is used with a subquery; for more
          information, see <a class="xref" href="sql-statements.html#derived-tables" title="13.2.11.8 Derived Tables">Section 13.2.11.8, “Derived Tables”</a>.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="explain-extended"></a>8.8.3 Extended EXPLAIN Output Format</h3>

</div>

</div>

</div>
<p>
        The <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> statement produces
        extra (<span class="quote">“<span class="quote">extended</span>”</span>) information that is not part of
        <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output but can be viewed
        by issuing a <a class="link" href="sql-statements.html#show-warnings" title="13.7.7.40 SHOW WARNINGS Statement"><code class="literal">SHOW WARNINGS</code></a>
        statement following <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>. As
        of MySQL 8.0.12, extended information is available for
        <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>,
        <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a>,
        <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>,
        <a class="link" href="sql-statements.html#replace" title="13.2.9 REPLACE Statement"><code class="literal">REPLACE</code></a>, and
        <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a> statements. Prior to
        8.0.12, extended information is available only for
        <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements.
      </p><p>
        The <code class="literal">Message</code> value in
        <a class="link" href="sql-statements.html#show-warnings" title="13.7.7.40 SHOW WARNINGS Statement"><code class="literal">SHOW WARNINGS</code></a> output displays how
        the optimizer qualifies table and column names in the
        <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statement, what the
        <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> looks like after the
        application of rewriting and optimization rules, and possibly
        other notes about the optimization process.
      </p><p>
        The extended information displayable with a
        <a class="link" href="sql-statements.html#show-warnings" title="13.7.7.40 SHOW WARNINGS Statement"><code class="literal">SHOW WARNINGS</code></a> statement following
        <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> is produced only for
        <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements.
        <a class="link" href="sql-statements.html#show-warnings" title="13.7.7.40 SHOW WARNINGS Statement"><code class="literal">SHOW WARNINGS</code></a> displays an empty
        result for other explainable statements
        (<a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a>,
        <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>,
        <a class="link" href="sql-statements.html#replace" title="13.2.9 REPLACE Statement"><code class="literal">REPLACE</code></a>, and
        <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a>).
      </p><p>
        Here is an example of extended
        <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN</code></strong>
       <strong class="userinput"><code>SELECT t1.a, t1.a IN (SELECT t2.a FROM t2) FROM t1\G</code></strong>
*************************** 1. row ***************************
           id: 1
  select_type: PRIMARY
        table: t1
         type: index
possible_keys: NULL
          key: PRIMARY
      key_len: 4
          ref: NULL
         rows: 4
     filtered: 100.00
        Extra: Using index
*************************** 2. row ***************************
           id: 2
  select_type: SUBQUERY
        table: t2
         type: index
possible_keys: a
          key: a
      key_len: 5
          ref: NULL
         rows: 3
     filtered: 100.00
        Extra: Using index
2 rows in set, 1 warning (0.00 sec)

mysql&gt; <strong class="userinput"><code>SHOW WARNINGS\G</code></strong>
*************************** 1. row ***************************
  Level: Note
   Code: 1003
Message: /* select#1 */ select `test`.`t1`.`a` AS `a`,
         &lt;in_optimizer&gt;(`test`.`t1`.`a`,`test`.`t1`.`a` in
         ( &lt;materialize&gt; (/* select#2 */ select `test`.`t2`.`a`
         from `test`.`t2` where 1 having 1 ),
         &lt;primary_index_lookup&gt;(`test`.`t1`.`a` in
         &lt;temporary table&gt; on &lt;auto_key&gt;
         where ((`test`.`t1`.`a` = `materialized-subquery`.`a`))))) AS `t1.a
         IN (SELECT t2.a FROM t2)` from `test`.`t1`
1 row in set (0.00 sec)
</pre><p>
        Because the statement displayed by <a class="link" href="sql-statements.html#show-warnings" title="13.7.7.40 SHOW WARNINGS Statement"><code class="literal">SHOW
        WARNINGS</code></a> may contain special markers to provide
        information about query rewriting or optimizer actions, the
        statement is not necessarily valid SQL and is not intended to be
        executed. The output may also include rows with
        <code class="literal">Message</code> values that provide additional
        non-SQL explanatory notes about actions taken by the optimizer.
      </p><p>
        The following list describes special markers that can appear in
        the extended output displayed by <a class="link" href="sql-statements.html#show-warnings" title="13.7.7.40 SHOW WARNINGS Statement"><code class="literal">SHOW
        WARNINGS</code></a>:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            <code class="literal">&lt;auto_key&gt;</code>
          </p><p>
            An automatically generated key for a temporary table.
          </p></li><li class="listitem"><p>
            <code class="literal">&lt;cache&gt;(<em class="replaceable"><code>expr</code></em>)</code>
          </p><p>
            The expression (such as a scalar subquery) is executed once
            and the resulting value is saved in memory for later use.
            For results consisting of multiple values, a temporary table
            may be created and you will see <code class="literal">&lt;temporary
            table&gt;</code> instead.
          </p></li><li class="listitem"><p>
            <code class="literal">&lt;exists&gt;(<em class="replaceable"><code>query
            fragment</code></em>)</code>
          </p><p>
            The subquery predicate is converted to an
            <code class="literal">EXISTS</code> predicate and the subquery is
            transformed so that it can be used together with the
            <code class="literal">EXISTS</code> predicate.
          </p></li><li class="listitem"><p>
            <code class="literal">&lt;in_optimizer&gt;(<em class="replaceable"><code>query
            fragment</code></em>)</code>
          </p><p>
            This is an internal optimizer object with no user
            significance.
          </p></li><li class="listitem"><p>
            <code class="literal">&lt;index_lookup&gt;(<em class="replaceable"><code>query
            fragment</code></em>)</code>
          </p><p>
            The query fragment is processed using an index lookup to
            find qualifying rows.
          </p></li><li class="listitem"><p>
            <code class="literal">&lt;if&gt;(<em class="replaceable"><code>condition</code></em>,
            <em class="replaceable"><code>expr1</code></em>,
            <em class="replaceable"><code>expr2</code></em>)</code>
          </p><p>
            If the condition is true, evaluate to
            <em class="replaceable"><code>expr1</code></em>, otherwise
            <em class="replaceable"><code>expr2</code></em>.
          </p></li><li class="listitem"><p>
            <code class="literal">&lt;is_not_null_test&gt;(<em class="replaceable"><code>expr</code></em>)</code>
          </p><p>
            A test to verify that the expression does not evaluate to
            <code class="literal">NULL</code>.
          </p></li><li class="listitem"><p>
            <code class="literal">&lt;materialize&gt;(<em class="replaceable"><code>query
            fragment</code></em>)</code>
          </p><p>
            Subquery materialization is used.
          </p></li><li class="listitem"><p>
            <code class="literal">`materialized-subquery`.<em class="replaceable"><code>col_name</code></em></code>
          </p><p>
            A reference to the column
            <em class="replaceable"><code>col_name</code></em> in an internal temporary
            table materialized to hold the result from evaluating a
            subquery.
          </p></li><li class="listitem"><p>
            <code class="literal">&lt;primary_index_lookup&gt;(<em class="replaceable"><code>query
            fragment</code></em>)</code>
          </p><p>
            The query fragment is processed using a primary key lookup
            to find qualifying rows.
          </p></li><li class="listitem"><p>
            <code class="literal">&lt;ref_null_helper&gt;(<em class="replaceable"><code>expr</code></em>)</code>
          </p><p>
            This is an internal optimizer object with no user
            significance.
          </p></li><li class="listitem"><p>
            <code class="literal">/* select#<em class="replaceable"><code>N</code></em> */
            <em class="replaceable"><code>select_stmt</code></em></code>
          </p><p>
            The <code class="literal">SELECT</code> is associated with the row in
            non-extended <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output
            that has an <code class="literal">id</code> value of
            <em class="replaceable"><code>N</code></em>.
          </p></li><li class="listitem"><p>
            <code class="literal"><em class="replaceable"><code>outer_tables</code></em> semi join
            (<em class="replaceable"><code>inner_tables</code></em>)</code>
          </p><p>
            A semijoin operation.
            <em class="replaceable"><code>inner_tables</code></em> shows the tables
            that were not pulled out. See <a class="xref" href="optimization.html#semijoins" title="8.2.2.1 Optimizing IN and EXISTS Subquery Predicates with Semijoin Transformations">Section 8.2.2.1, “Optimizing IN and EXISTS Subquery Predicates with Semijoin
          Transformations”</a>.
          </p></li><li class="listitem"><p>
            <code class="literal">&lt;temporary table&gt;</code>
          </p><p>
            This represents an internal temporary table created to cache
            an intermediate result.
</p></li></ul>
</div>
<p>
        When some tables are of <a class="link" href="optimization.html#jointype_const"><code class="literal">const</code></a>
        or <a class="link" href="optimization.html#jointype_system"><code class="literal">system</code></a> type, expressions
        involving columns from these tables are evaluated early by the
        optimizer and are not part of the displayed statement. However,
        with <code class="literal">FORMAT=JSON</code>, some
        <a class="link" href="optimization.html#jointype_const"><code class="literal">const</code></a> table accesses are
        displayed as a <a class="link" href="optimization.html#jointype_ref"><code class="literal">ref</code></a> access
        that uses a const value.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="explain-for-connection"></a>8.8.4 Obtaining Execution Plan Information for a Named Connection</h3>

</div>

</div>

</div>
<p>
        To obtain the execution plan for an explainable statement
        executing in a named connection, use this statement:
      </p><pre data-lang="sql" class="programlisting">EXPLAIN [<em class="replaceable"><code>options</code></em>] FOR CONNECTION <em class="replaceable"><code>connection_id</code></em>;
</pre><p>
        <a class="link" href="optimization.html#explain-for-connection" title="8.8.4 Obtaining Execution Plan Information for a Named Connection"><code class="literal">EXPLAIN FOR CONNECTION</code></a> returns
        the <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> information that is
        currently being used to execute a query in a given connection.
        Because of changes to data (and supporting statistics) it may
        produce a different result from running
        <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> on the equivalent query
        text. This difference in behavior can be useful in diagnosing
        more transient performance problems. For example, if you are
        running a statement in one session that is taking a long time to
        complete, using <a class="link" href="optimization.html#explain-for-connection" title="8.8.4 Obtaining Execution Plan Information for a Named Connection"><code class="literal">EXPLAIN FOR
        CONNECTION</code></a> in another session may yield useful
        information about the cause of the delay.
      </p><p>
        <em class="replaceable"><code>connection_id</code></em> is the connection
        identifier, as obtained from the
        <code class="literal">INFORMATION_SCHEMA</code>
        <a class="link" href="information-schema.html#processlist-table" title="25.22 The INFORMATION_SCHEMA PROCESSLIST Table"><code class="literal">PROCESSLIST</code></a> table or the
        <a class="link" href="sql-statements.html#show-processlist" title="13.7.7.29 SHOW PROCESSLIST Statement"><code class="literal">SHOW PROCESSLIST</code></a> statement. If
        you have the <a class="link" href="security.html#priv_process"><code class="literal">PROCESS</code></a> privilege,
        you can specify the identifier for any connection. Otherwise,
        you can specify the identifier only for your own connections. In
        all cases, you must have sufficient privileges to explain the
        query on the specified connection.
      </p><p>
        If the named connection is not executing a statement, the result
        is empty. Otherwise, <code class="literal">EXPLAIN FOR CONNECTION</code>
        applies only if the statement being executed in the named
        connection is explainable. This includes
        <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>,
        <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a>,
        <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>,
        <a class="link" href="sql-statements.html#replace" title="13.2.9 REPLACE Statement"><code class="literal">REPLACE</code></a>, and
        <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a>. (However,
        <code class="literal">EXPLAIN FOR CONNECTION</code> does not work for
        prepared statements, even prepared statements of those types.)
      </p><p>
        If the named connection is executing an explainable statement,
        the output is what you would obtain by using
        <code class="literal">EXPLAIN</code> on the statement itself.
      </p><p>
        If the named connection is executing a statement that is not
        explainable, an error occurs. For example, you cannot name the
        connection identifier for your current session because
        <code class="literal">EXPLAIN</code> is not explainable:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT CONNECTION_ID();</code></strong>
+-----------------+
| CONNECTION_ID() |
+-----------------+
|             373 |
+-----------------+
1 row in set (0.00 sec)

mysql&gt; <strong class="userinput"><code>EXPLAIN FOR CONNECTION 373;</code></strong>
ERROR 1889 (HY000): EXPLAIN FOR CONNECTION command is supported
only for SELECT/UPDATE/INSERT/DELETE/REPLACE
</pre><p>
        The <code class="literal">Com_explain_other</code> status variable
        indicates the number of
        <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN FOR
        CONNECTION</code></a> statements executed.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="estimating-performance"></a>8.8.5 Estimating Query Performance</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444348052896"></a><a class="indexterm" name="idm46444348051440"></a><a class="indexterm" name="idm46444348049952"></a><p>
        In most cases, you can estimate query performance by counting
        disk seeks. For small tables, you can usually find a row in one
        disk seek (because the index is probably cached). For bigger
        tables, you can estimate that, using B-tree indexes, you need
        this many seeks to find a row:
        <code class="literal">log(<em class="replaceable"><code>row_count</code></em>) /
        log(<em class="replaceable"><code>index_block_length</code></em> / 3 * 2 /
        (<em class="replaceable"><code>index_length</code></em> +
        <em class="replaceable"><code>data_pointer_length</code></em>)) + 1</code>.
      </p><p>
        In MySQL, an index block is usually 1,024 bytes and the data
        pointer is usually four bytes. For a 500,000-row table with a
        key value length of three bytes (the size of
        <a class="link" href="data-types.html#integer-types" title="11.1.2 Integer Types (Exact Value) - INTEGER, INT, SMALLINT, TINYINT, MEDIUMINT, BIGINT"><code class="literal">MEDIUMINT</code></a>), the formula indicates
        <code class="literal">log(500,000)/log(1024/3*2/(3+4)) + 1</code> =
        <code class="literal">4</code> seeks.
      </p><p>
        This index would require storage of about 500,000 * 7 * 3/2 =
        5.2MB (assuming a typical index buffer fill ratio of 2/3), so
        you probably have much of the index in memory and so need only
        one or two calls to read data to find the row.
      </p><p>
        For writes, however, you need four seek requests to find where
        to place a new index value and normally two seeks to update the
        index and write the row.
      </p><p>
        The preceding discussion does not mean that your application
        performance slowly degenerates by log
        <em class="replaceable"><code>N</code></em>. As long as everything is cached by
        the OS or the MySQL server, things become only marginally slower
        as the table gets bigger. After the data gets too big to be
        cached, things start to go much slower until your applications
        are bound only by disk seeks (which increase by log
        <em class="replaceable"><code>N</code></em>). To avoid this, increase the key
        cache size as the data grows. For <code class="literal">MyISAM</code>
        tables, the key cache size is controlled by the
        <a class="link" href="server-administration.html#sysvar_key_buffer_size"><code class="literal">key_buffer_size</code></a> system
        variable. See <a class="xref" href="server-administration.html#server-configuration" title="5.1.1 Configuring the Server">Section 5.1.1, “Configuring the Server”</a>.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a name="controlling-optimizer"></a>8.9 Controlling the Query Optimizer</h2>

</div>

</div>

</div>
<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#controlling-query-plan-evaluation">8.9.1 Controlling Query Plan Evaluation</a></span></dt><dt><span class="section"><a href="optimization.html#switchable-optimizations">8.9.2 Switchable Optimizations</a></span></dt><dt><span class="section"><a href="optimization.html#optimizer-hints">8.9.3 Optimizer Hints</a></span></dt><dt><span class="section"><a href="optimization.html#index-hints">8.9.4 Index Hints</a></span></dt><dt><span class="section"><a href="optimization.html#cost-model">8.9.5 The Optimizer Cost Model</a></span></dt><dt><span class="section"><a href="optimization.html#optimizer-statistics">8.9.6 Optimizer Statistics</a></span></dt></dl>
</div>
<a class="indexterm" name="idm46444348034880"></a><p>
      MySQL provides optimizer control through system variables that
      affect how query plans are evaluated, switchable optimizations,
      optimizer and index hints, and the optimizer cost model.
    </p><p>
      The server also maintains statistics about column values, although
      the optimizer does not yet use this information.
</p>
<div class="section">

<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="controlling-query-plan-evaluation"></a>8.9.1 Controlling Query Plan Evaluation</h3>
</div>
</div>
</div>
<a class="indexterm" name="idm46444348031088"></a><p>
        The task of the query optimizer is to find an optimal plan for
        executing an SQL query. Because the difference in performance
        between <span class="quote">“<span class="quote">good</span>”</span> and <span class="quote">“<span class="quote">bad</span>”</span> plans can be
        orders of magnitude (that is, seconds versus hours or even
        days), most query optimizers, including that of MySQL, perform a
        more or less exhaustive search for an optimal plan among all
        possible query evaluation plans. For join queries, the number of
        possible plans investigated by the MySQL optimizer grows
        exponentially with the number of tables referenced in a query.
        For small numbers of tables (typically less than 7 to 10) this
        is not a problem. However, when larger queries are submitted,
        the time spent in query optimization may easily become the major
        bottleneck in the server's performance.
      </p><p>
        A more flexible method for query optimization enables the user
        to control how exhaustive the optimizer is in its search for an
        optimal query evaluation plan. The general idea is that the
        fewer plans that are investigated by the optimizer, the less
        time it spends in compiling a query. On the other hand, because
        the optimizer skips some plans, it may miss finding an optimal
        plan.
      </p><p>
        The behavior of the optimizer with respect to the number of
        plans it evaluates can be controlled using two system variables:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            The <a class="link" href="server-administration.html#sysvar_optimizer_prune_level"><code class="literal">optimizer_prune_level</code></a>
            variable tells the optimizer to skip certain plans based on
            estimates of the number of rows accessed for each table. Our
            experience shows that this kind of <span class="quote">“<span class="quote">educated
            guess</span>”</span> rarely misses optimal plans, and may
            dramatically reduce query compilation times. That is why
            this option is on
            (<code class="literal">optimizer_prune_level=1</code>) by default.
            However, if you believe that the optimizer missed a better
            query plan, this option can be switched off
            (<code class="literal">optimizer_prune_level=0</code>) with the risk
            that query compilation may take much longer. Note that, even
            with the use of this heuristic, the optimizer still explores
            a roughly exponential number of plans.
          </p></li><li class="listitem"><p>
            The <a class="link" href="server-administration.html#sysvar_optimizer_search_depth"><code class="literal">optimizer_search_depth</code></a>
            variable tells how far into the <span class="quote">“<span class="quote">future</span>”</span> of
            each incomplete plan the optimizer should look to evaluate
            whether it should be expanded further. Smaller values of
            <a class="link" href="server-administration.html#sysvar_optimizer_search_depth"><code class="literal">optimizer_search_depth</code></a> may
            result in orders of magnitude smaller query compilation
            times. For example, queries with 12, 13, or more tables may
            easily require hours and even days to compile if
            <a class="link" href="server-administration.html#sysvar_optimizer_search_depth"><code class="literal">optimizer_search_depth</code></a> is
            close to the number of tables in the query. At the same
            time, if compiled with
            <a class="link" href="server-administration.html#sysvar_optimizer_search_depth"><code class="literal">optimizer_search_depth</code></a>
            equal to 3 or 4, the optimizer may compile in less than a
            minute for the same query. If you are unsure of what a
            reasonable value is for
            <a class="link" href="server-administration.html#sysvar_optimizer_search_depth"><code class="literal">optimizer_search_depth</code></a>,
            this variable can be set to 0 to tell the optimizer to
            determine the value automatically.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="switchable-optimizations"></a>8.9.2 Switchable Optimizations</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444348012192"></a><a class="indexterm" name="idm46444348011104"></a><p>
        The <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
        variable enables control over optimizer behavior. Its value is a
        set of flags, each of which has a value of <code class="literal">on</code>
        or <code class="literal">off</code> to indicate whether the corresponding
        optimizer behavior is enabled or disabled. This variable has
        global and session values and can be changed at runtime. The
        global default can be set at server startup.
      </p><p>
        To see the current set of optimizer flags, select the variable
        value:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT @@optimizer_switch\G</code></strong>
*************************** 1. row ***************************
@@optimizer_switch: index_merge=on,index_merge_union=on,
                    index_merge_sort_union=on,index_merge_intersection=on,
                    engine_condition_pushdown=on,index_condition_pushdown=on,
                    mrr=on,mrr_cost_based=on,block_nested_loop=on,
                    batched_key_access=off,materialization=on,semijoin=on,
                    loosescan=on,firstmatch=on,duplicateweedout=on,
                    subquery_materialization_cost_based=on,
                    use_index_extensions=on,condition_fanout_filter=on,
                    derived_merge=on,use_invisible_indexes=off,skip_scan=on,
                    hash_join=on
</pre><p>
        To change the value of
        <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a>, assign a
        value consisting of a comma-separated list of one or more
        commands:
      </p><pre data-lang="sql" class="programlisting">SET [GLOBAL|SESSION] optimizer_switch='<em class="replaceable"><code>command</code></em>[,<em class="replaceable"><code>command</code></em>]...';
</pre><p>
        Each <em class="replaceable"><code>command</code></em> value should have one of
        the forms shown in the following table.
</p>
<div class="informaltable">
<table summary="The syntax of the command value for SET optimizer_switch commands."><col width="25%"><col width="75%"><thead><tr>
            <th scope="col">Command Syntax</th>
            <th scope="col">Meaning</th>
          </tr></thead><tbody><tr>
            <td scope="row"><code class="literal">default</code></td>
            <td>Reset every optimization to its default value</td>
          </tr><tr>
            <td scope="row"><code class="literal"><em class="replaceable"><code>opt_name</code></em>=default</code></td>
            <td>Set the named optimization to its default value</td>
          </tr><tr>
            <td scope="row"><code class="literal"><em class="replaceable"><code>opt_name</code></em>=off</code></td>
            <td>Disable the named optimization</td>
          </tr><tr>
            <td scope="row"><code class="literal"><em class="replaceable"><code>opt_name</code></em>=on</code></td>
            <td>Enable the named optimization</td>
</tr></tbody></table>
</div>
<p>
        The order of the commands in the value does not matter, although
        the <code class="literal">default</code> command is executed first if
        present. Setting an <em class="replaceable"><code>opt_name</code></em> flag to
        <code class="literal">default</code> sets it to whichever of
        <code class="literal">on</code> or <code class="literal">off</code> is its default
        value. Specifying any given <em class="replaceable"><code>opt_name</code></em>
        more than once in the value is not permitted and causes an
        error. Any errors in the value cause the assignment to fail with
        an error, leaving the value of
        <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> unchanged.
      </p><p>
        The following list describes the permissible
        <em class="replaceable"><code>opt_name</code></em> flag names, grouped by
        optimization strategy:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Batched Key Access Flags
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">batched_key_access</code> (default
                <code class="literal">off</code>)
              </p><p>
                Controls use of BKA join algorithm.
</p></li></ul>
</div>
<p>
            For <code class="literal">batched_key_access</code> to have any effect
            when set to <code class="literal">on</code>, the
            <code class="literal">mrr</code> flag must also be
            <code class="literal">on</code>. Currently, the cost estimation for
            MRR is too pessimistic. Hence, it is also necessary for
            <code class="literal">mrr_cost_based</code> to be
            <code class="literal">off</code> for BKA to be used.
          </p><p>
            For more information, see
            <a class="xref" href="optimization.html#bnl-bka-optimization" title="8.2.1.12 Block Nested-Loop and Batched Key Access Joins">Section 8.2.1.12, “Block Nested-Loop and Batched Key Access Joins”</a>.
          </p></li><li class="listitem"><p>
            Block Nested-Loop Flags
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">block_nested_loop</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls use of BNL join algorithm.
</p></li></ul>
</div>
<p>
            For more information, see
            <a class="xref" href="optimization.html#bnl-bka-optimization" title="8.2.1.12 Block Nested-Loop and Batched Key Access Joins">Section 8.2.1.12, “Block Nested-Loop and Batched Key Access Joins”</a>.
          </p></li><li class="listitem"><p>
            Condition Filtering Flags
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">condition_fanout_filter</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls use of condition filtering.
</p></li></ul>
</div>
<p>
            For more information, see
            <a class="xref" href="optimization.html#condition-filtering" title="8.2.1.13 Condition Filtering">Section 8.2.1.13, “Condition Filtering”</a>.
          </p></li><li class="listitem"><p>
            Derived Table Merging Flags
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">derived_merge</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls merging of derived tables and views into outer
                query block.
</p></li></ul>
</div>
<p>
            The <code class="literal">derived_merge</code> flag controls whether
            the optimizer attempts to merge derived tables, view
            references, and common table expressions into the outer
            query block, assuming that no other rule prevents merging;
            for example, an <code class="literal">ALGORITHM</code> directive for a
            view takes precedence over the
            <code class="literal">derived_merge</code> setting. By default, the
            flag is <code class="literal">on</code> to enable merging.
          </p><p>
            For more information, see
            <a class="xref" href="optimization.html#derived-table-optimization" title="8.2.2.4 Optimizing Derived Tables, View References, and Common Table Expressions with Merging or Materialization">Section 8.2.2.4, “Optimizing Derived Tables, View References, and Common Table Expressions
          with Merging or Materialization”</a>.
          </p></li><li class="listitem"><p>
            Engine Condition Pushdown Flags
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">engine_condition_pushdown</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls engine condition pushdown.
</p></li></ul>
</div>
<p>
            For more information, see
            <a class="xref" href="optimization.html#condition-pushdown-optimization" title="8.2.1.5 Engine Condition Pushdown Optimization">Section 8.2.1.5, “Engine Condition Pushdown Optimization”</a>.
          </p></li><li class="listitem"><p>
            Hash Join Flags
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">hash_join</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls hash joins (MySQL 8.0.18 only; has no effect in
                MySQL 8.0.19 or later).
</p></li></ul>
</div>
<p>
            For more information, see <a class="xref" href="optimization.html#hash-joins" title="8.2.1.4 Hash Join Optimization">Section 8.2.1.4, “Hash Join Optimization”</a>.
          </p></li><li class="listitem"><p>
            Index Condition Pushdown Flags
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">index_condition_pushdown</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls index condition pushdown.
</p></li></ul>
</div>
<p>
            For more information, see
            <a class="xref" href="optimization.html#index-condition-pushdown-optimization" title="8.2.1.6 Index Condition Pushdown Optimization">Section 8.2.1.6, “Index Condition Pushdown Optimization”</a>.
          </p></li><li class="listitem"><p>
            Index Extensions Flags
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">use_index_extensions</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls use of index extensions.
</p></li></ul>
</div>
<p>
            For more information, see
            <a class="xref" href="optimization.html#index-extensions" title="8.3.10 Use of Index Extensions">Section 8.3.10, “Use of Index Extensions”</a>.
          </p></li><li class="listitem"><p>
            Index Merge Flags
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">index_merge</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls all Index Merge optimizations.
              </p></li><li class="listitem"><p>
                <code class="literal">index_merge_intersection</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls the Index Merge Intersection Access
                optimization.
              </p></li><li class="listitem"><p>
                <code class="literal">index_merge_sort_union</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls the Index Merge Sort-Union Access optimization.
              </p></li><li class="listitem"><p>
                <code class="literal">index_merge_union</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls the Index Merge Union Access optimization.
</p></li></ul>
</div>
<p>
            For more information, see
            <a class="xref" href="optimization.html#index-merge-optimization" title="8.2.1.3 Index Merge Optimization">Section 8.2.1.3, “Index Merge Optimization”</a>.
          </p></li><li class="listitem"><p>
            Index Visibility Flags
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">use_invisible_indexes</code> (default
                <code class="literal">off</code>)
              </p><p>
                Controls use of invisible indexes.
</p></li></ul>
</div>
<p>
            For more information, see
            <a class="xref" href="optimization.html#invisible-indexes" title="8.3.12 Invisible Indexes">Section 8.3.12, “Invisible Indexes”</a>.
          </p></li><li class="listitem"><p>
            Multi-Range Read Flags
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">mrr</code> (default <code class="literal">on</code>)
              </p><p>
                Controls the Multi-Range Read strategy.
              </p></li><li class="listitem"><p>
                <code class="literal">mrr_cost_based</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls use of cost-based MRR if
                <code class="literal">mrr=on</code>.
</p></li></ul>
</div>
<p>
            For more information, see
            <a class="xref" href="optimization.html#mrr-optimization" title="8.2.1.11 Multi-Range Read Optimization">Section 8.2.1.11, “Multi-Range Read Optimization”</a>.
          </p></li><li class="listitem"><p>
            Skip Scan Flags
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">skip_scan</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls use of Skip Scan access method.
</p></li></ul>
</div>
<p>
            For more information, see
            <a class="xref" href="optimization.html#range-access-skip-scan" title="Skip Scan Range Access Method">Skip Scan Range Access Method</a>.
          </p></li><li class="listitem"><p>
            Semijoin Flags
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">semijoin</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls all semijoin strategies.
              </p><p>
                In MySQL 8.0.17 and later, this also applies to the
                antijoin optimization.
              </p></li><li class="listitem"><p>
                <code class="literal">duplicateweedout</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls the semijoin Duplicate Weedout strategy.
              </p></li><li class="listitem"><p>
                <code class="literal">firstmatch</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls the semijoin FirstMatch strategy.
              </p></li><li class="listitem"><p>
                <code class="literal">loosescan</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls the semijoin LooseScan strategy (not to be
                confused with Loose Index Scan for <code class="literal">GROUP
                BY</code>).
</p></li></ul>
</div>
<p>
            The <code class="literal">semijoin</code>,
            <code class="literal">firstmatch</code>, <code class="literal">loosescan</code>,
            and <code class="literal">duplicateweedout</code> flags enable control
            over semijoin strategies. The <code class="literal">semijoin</code>
            flag controls whether semijoins are used. If it is set to
            <code class="literal">on</code>, the <code class="literal">firstmatch</code> and
            <code class="literal">loosescan</code> flags enable finer control over
            the permitted semijoin strategies.
          </p><p>
            If the <code class="literal">duplicateweedout</code> semijoin strategy
            is disabled, it is not used unless all other applicable
            strategies are also disabled.
          </p><p>
            If <code class="literal">semijoin</code> and
            <code class="literal">materialization</code> are both
            <code class="literal">on</code>, semijoins also use materialization
            where applicable. These flags are <code class="literal">on</code> by
            default.
          </p><p>
            For more information, see <a class="xref" href="optimization.html#semijoins" title="8.2.2.1 Optimizing IN and EXISTS Subquery Predicates with Semijoin Transformations">Section 8.2.2.1, “Optimizing IN and EXISTS Subquery Predicates with Semijoin
          Transformations”</a>.
          </p></li><li class="listitem"><p>
            Subquery Materialization Flags
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">materialization</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls materialization (including semijoin
                materialization).
              </p></li><li class="listitem"><p>
                <code class="literal">subquery_materialization_cost_based</code>
                (default <code class="literal">on</code>)
              </p><p>
                Use cost-based materialization choice.
</p></li></ul>
</div>
<p>
            The <code class="literal">materialization</code> flag controls whether
            subquery materialization is used. If
            <code class="literal">semijoin</code> and
            <code class="literal">materialization</code> are both
            <code class="literal">on</code>, semijoins also use materialization
            where applicable. These flags are <code class="literal">on</code> by
            default.
          </p><p>
            The <code class="literal">subquery_materialization_cost_based</code>
            flag enables control over the choice between subquery
            materialization and
            <code class="literal">IN</code>-to-<code class="literal">EXISTS</code> subquery
            transformation. If the flag is <code class="literal">on</code> (the
            default), the optimizer performs a cost-based choice between
            subquery materialization and
            <code class="literal">IN</code>-to-<code class="literal">EXISTS</code> subquery
            transformation if either method could be used. If the flag
            is <code class="literal">off</code>, the optimizer chooses subquery
            materialization over
            <code class="literal">IN</code>-to-<code class="literal">EXISTS</code> subquery
            transformation.
          </p><p>
            For more information, see
            <a class="xref" href="optimization.html#subquery-optimization" title="8.2.2 Optimizing Subqueries, Derived Tables, View References, and Common Table Expressions">Section 8.2.2, “Optimizing Subqueries, Derived Tables, View References, and Common Table
        Expressions”</a>.
          </p></li><li class="listitem"><p>
            Limit Optimization Flags
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">prefer_ordering_index</code> (default
                <code class="literal">on</code>)
              </p><p>
                Controls whether, in the case of a query having an
                <code class="literal">ORDER BY</code> or <code class="literal">GROUP
                BY</code> with a <code class="literal">LIMIT</code> clause, the
                optimizer tries to use an ordered index instead of an
                unordered index, a filesort, or some other optimization.
                This optimzation
                (<code class="literal">reconsidering_access_paths_for_index_ordering</code>)
                is performed by default.
              </p><p>
                There are cases, particularly when the
                <code class="literal">LIMIT</code> value is relatively large, in
                which this may not be desirable, but, prior to MySQL
                8.0.21, it ws not possible to disable this optimization.
                In MySQL 8.0.21 and later, it remains the default
                behavior, but can be disabled by setting the
                <code class="literal">prefer_ordering_index</code> flag to
                <code class="literal">off</code>.
</p></li></ul>
</div>
<p>
            For more information, see
            <a class="xref" href="optimization.html#limit-optimization" title="8.2.1.19 LIMIT Query Optimization">Section 8.2.1.19, “LIMIT Query Optimization”</a>.
</p></li></ul>
</div>
<p>
        When you assign a value to
        <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a>, flags that
        are not mentioned keep their current values. This makes it
        possible to enable or disable specific optimizer behaviors in a
        single statement without affecting other behaviors. The
        statement does not depend on what other optimizer flags exist
        and what their values are. Suppose that all Index Merge
        optimizations are enabled:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT @@optimizer_switch\G</code></strong>
*************************** 1. row ***************************
@@optimizer_switch: index_merge=on,index_merge_union=on,
                    index_merge_sort_union=on,
                    index_merge_intersection=on,
                    engine_condition_pushdown=on,
                    index_condition_pushdown=on,
                    mrr=on,mrr_cost_based=on,
                    block_nested_loop=on,batched_key_access=off,
                    materialization=on,semijoin=on,loosescan=on,
                    firstmatch=on,
                    subquery_materialization_cost_based=on,
                    use_index_extensions=on,
                    condition_fanout_filter=on
</pre><p>
        If the server is using the Index Merge Union or Index Merge
        Sort-Union access methods for certain queries and you want to
        check whether the optimizer will perform better without them,
        set the variable value like this:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SET optimizer_switch='index_merge_union=off,index_merge_sort_union=off';</code></strong>

mysql&gt; <strong class="userinput"><code>SELECT @@optimizer_switch\G</code></strong>
*************************** 1. row ***************************
@@optimizer_switch: index_merge=on,index_merge_union=off,
                    index_merge_sort_union=off,
                    index_merge_intersection=on,
                    engine_condition_pushdown=on,
                    index_condition_pushdown=on,
                    mrr=on,mrr_cost_based=on,
                    block_nested_loop=on,batched_key_access=off,
                    materialization=on,semijoin=on,loosescan=on,
                    firstmatch=on,
                    subquery_materialization_cost_based=on,
                    use_index_extensions=on,
                    condition_fanout_filter=on
</pre>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="optimizer-hints"></a>8.9.3 Optimizer Hints</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444347829056"></a><a class="indexterm" name="idm46444347827984"></a><p>
        One means of control over optimizer strategies is to set the
        <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
        variable (see <a class="xref" href="optimization.html#switchable-optimizations" title="8.9.2 Switchable Optimizations">Section 8.9.2, “Switchable Optimizations”</a>).
        Changes to this variable affect execution of all subsequent
        queries; to affect one query differently from another, it is
        necessary to change
        <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> before each
        one.
      </p><p>
        Another way to control the optimizer is by using optimizer
        hints, which can be specified within individual statements.
        Because optimizer hints apply on a per-statement basis, they
        provide finer control over statement execution plans than can be
        achieved using
        <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a>. For example,
        you can enable an optimization for one table in a statement and
        disable the optimization for a different table. Hints within a
        statement take precedence over
        <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> flags.
      </p><p>
        Examples:
      </p><pre data-lang="sql" class="programlisting">SELECT /*+ NO_RANGE_OPTIMIZATION(t3 PRIMARY, f2_idx) */ f1
  FROM t3 WHERE f1 &gt; 30 AND f1 &lt; 33;
SELECT /*+ BKA(t1) NO_BKA(t2) */ * FROM t1 INNER JOIN t2 WHERE ...;
SELECT /*+ NO_ICP(t1, t2) */ * FROM t1 INNER JOIN t2 WHERE ...;
SELECT /*+ SEMIJOIN(FIRSTMATCH, LOOSESCAN) */ * FROM t1 ...;
EXPLAIN SELECT /*+ NO_ICP(t1) */ * FROM t1 WHERE ...;
SELECT /*+ MERGE(dt) */ * FROM (SELECT * FROM t1) AS dt;
INSERT /*+ SET_VAR(foreign_key_checks=OFF) */ INTO t2 VALUES(2);</pre><p>
        Optimizer hints, described here, differ from index hints,
        described in <a class="xref" href="optimization.html#index-hints" title="8.9.4 Index Hints">Section 8.9.4, “Index Hints”</a>. Optimizer and index
        hints may be used separately or together.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a class="xref" href="optimization.html#optimizer-hints-overview" title="Optimizer Hint Overview">Optimizer Hint Overview</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#optimizer-hints-syntax" title="Optimizer Hint Syntax">Optimizer Hint Syntax</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints">Join-Order Optimizer Hints</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints">Table-Level Optimizer Hints</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints">Index-Level Optimizer Hints</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#optimizer-hints-subquery" title="Subquery Optimizer Hints">Subquery Optimizer Hints</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#optimizer-hints-execution-time" title="Statement Execution Time Optimizer Hints">Statement Execution Time Optimizer Hints</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#optimizer-hints-set-var" title="Variable-Setting Hint Syntax">Variable-Setting Hint Syntax</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#optimizer-hints-resource-group" title="Resource Group Hint Syntax">Resource Group Hint Syntax</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#optimizer-hints-query-block-naming" title="Optimizer Hints for Naming Query Blocks">Optimizer Hints for Naming Query Blocks</a></p></li></ul>
</div>

<div class="simplesect">

<div class="titlepage">
<div>

<div class="simple">
<h4 class="title"><a name="optimizer-hints-overview"></a>Optimizer Hint Overview</h4>

</div>

</div>

</div>
<p>
          Optimizer hints apply at different scope levels:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Global: The hint affects the entire statement
            </p></li><li class="listitem"><p>
              Query block: The hint affects a particular query block
              within a statement
            </p></li><li class="listitem"><p>
              Table-level: The hint affects a particular table within a
              query block
            </p></li><li class="listitem"><p>
              Index-level: The hint affects a particular index within a
              table
</p></li></ul>
</div>
<p>
          The following table summarizes the available optimizer hints,
          the optimizer strategies they affect, and the scope or scopes
          at which they apply. More details are given later.
</p>
<div class="table">
<a name="optimizer-hints-table"></a><p class="title"><b>Table 8.2 Optimizer Hints Available</b></p>
<div class="table-contents">
<table summary="Optimizer hint names, descriptions, and contexts in which they apply."><col width="30%"><col width="40%"><col width="30%"><thead><tr>
              <th scope="col">Hint Name</th>
              <th scope="col">Description</th>
              <th scope="col">Applicable Scopes</th>
            </tr></thead><tbody><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">BKA</code></a>,
                <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">NO_BKA</code></a></td>
              <td align="left">Affects Batched Key Access join processing</td>
              <td>Query block, table</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">BNL</code></a>,
                <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">NO_BNL</code></a></td>
              <td align="left">Affects Block Nested-Loop join processing</td>
              <td>Query block, table</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">HASH_JOIN</code></a>,
                <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">NO_HASH_JOIN</code></a></td>
              <td align="left">Affects Hash Join optimization</td>
              <td>Query block, table</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">INDEX_MERGE</code></a>,
                <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_INDEX_MERGE</code></a></td>
              <td align="left">Affects Index Merge optimization</td>
              <td>Table, index</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_FIXED_ORDER</code></a></td>
              <td align="left">Use table order specified in <code class="literal">FROM</code> clause for join
                order</td>
              <td>Query block</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_ORDER</code></a></td>
              <td align="left">Use table order specified in hint for join order</td>
              <td>Query block</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_PREFIX</code></a></td>
              <td align="left">Use table order specified in hint for first tables of join order</td>
              <td>Query block</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_SUFFIX</code></a></td>
              <td align="left">Use table order specified in hint for last tables of join order</td>
              <td>Query block</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-execution-time" title="Statement Execution Time Optimizer Hints"><code class="literal">MAX_EXECUTION_TIME</code></a></td>
              <td align="left">Limits statement execution time</td>
              <td>Global</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">MERGE</code></a>,
                <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">NO_MERGE</code></a></td>
              <td align="left">Affects derived table/view merging into outer query block</td>
              <td>Table</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">MRR</code></a>,
                <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_MRR</code></a></td>
              <td align="left">Affects Multi-Range Read optimization</td>
              <td>Table, index</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_ICP</code></a></td>
              <td align="left">Affects Index Condition Pushdown optimization</td>
              <td>Table, index</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_RANGE_OPTIMIZATION</code></a></td>
              <td align="left">Affects range optimization</td>
              <td>Table, index</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-query-block-naming" title="Optimizer Hints for Naming Query Blocks"><code class="literal">QB_NAME</code></a></td>
              <td align="left">Assigns name to query block</td>
              <td>Query block</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-resource-group" title="Resource Group Hint Syntax"><code class="literal">RESOURCE_GROUP</code></a></td>
              <td align="left">Set resource group during statement execution</td>
              <td>Global</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-subquery" title="Subquery Optimizer Hints"><code class="literal">SEMIJOIN</code></a>,
                <a class="link" href="optimization.html#optimizer-hints-subquery" title="Subquery Optimizer Hints"><code class="literal">NO_SEMIJOIN</code></a></td>
              <td align="left">Affects semijoin strategies; beginning with MySQL 8.0.17, this also
                applies to antijoins</td>
              <td>Query block</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">SKIP_SCAN</code></a>,
                <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_SKIP_SCAN</code></a></td>
              <td align="left">Affects Skip Scan optimization</td>
              <td>Table, index</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-set-var" title="Variable-Setting Hint Syntax"><code class="literal">SET_VAR</code></a></td>
              <td align="left">Set variable during statement execution</td>
              <td>Global</td>
            </tr><tr>
              <td align="left" scope="row"><a class="link" href="optimization.html#optimizer-hints-subquery" title="Subquery Optimizer Hints"><code class="literal">SUBQUERY</code></a></td>
              <td align="left">Affects materialization,
                <code class="literal">IN</code>-to-<code class="literal">EXISTS</code>
                subquery stratgies</td>
              <td>Query block</td>
</tr></tbody></table>
</div>

</div>
<br class="table-break"><p>
          Disabling an optimization prevents the optimizer from using
          it. Enabling an optimization means the optimizer is free to
          use the strategy if it applies to statement execution, not
          that the optimizer necessarily will use it.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="optimizer-hints-syntax"></a>Optimizer Hint Syntax</h4>

</div>

</div>

</div>
<p>
          MySQL supports comments in SQL statements as described in
          <a class="xref" href="language-structure.html#comments" title="9.6 Comment Syntax">Section 9.6, “Comment Syntax”</a>. Optimizer hints must be specified
          within <code class="literal">/*+ ... */</code> comments. That is,
          optimizer hints use a variant of <code class="literal">/* ... */</code>
          C-style comment syntax, with a <code class="literal">+</code> character
          following the <code class="literal">/*</code> comment opening sequence.
          Examples:
        </p><pre data-lang="sql" class="programlisting">/*+ BKA(t1) */
/*+ BNL(t1, t2) */
/*+ NO_RANGE_OPTIMIZATION(t4 PRIMARY) */
/*+ QB_NAME(qb2) */</pre><p>
          Whitespace is permitted after the <code class="literal">+</code>
          character.
        </p><p>
          The parser recognizes optimizer hint comments after the
          initial keyword of <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>,
          <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a>,
          <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>,
          <a class="link" href="sql-statements.html#replace" title="13.2.9 REPLACE Statement"><code class="literal">REPLACE</code></a>, and
          <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a> statements. Hints are
          permitted in these contexts:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              At the beginning of query and data change statements:
            </p><pre data-lang="sql" class="programlisting">SELECT /*+ ... */ ...
INSERT /*+ ... */ ...
REPLACE /*+ ... */ ...
UPDATE /*+ ... */ ...
DELETE /*+ ... */ ...</pre></li><li class="listitem"><p>
              At the beginning of query blocks:
            </p><pre data-lang="sql" class="programlisting">(SELECT /*+ ... */ ... )
(SELECT ... ) UNION (SELECT /*+ ... */ ... )
(SELECT /*+ ... */ ... ) UNION (SELECT /*+ ... */ ... )
UPDATE ... WHERE x IN (SELECT /*+ ... */ ...)
INSERT ... SELECT /*+ ... */ ...</pre></li><li class="listitem"><p>
              In hintable statements prefaced by
              <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>. For example:
            </p><pre data-lang="sql" class="programlisting">EXPLAIN SELECT /*+ ... */ ...
EXPLAIN UPDATE ... WHERE x IN (SELECT /*+ ... */ ...)</pre><p>
              The implication is that you can use
              <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> to see how
              optimizer hints affect execution plans. Use
              <a class="link" href="sql-statements.html#show-warnings" title="13.7.7.40 SHOW WARNINGS Statement"><code class="literal">SHOW WARNINGS</code></a> immediately
              after <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> to see how
              hints are used. The extended <code class="literal">EXPLAIN</code>
              output displayed by a following <a class="link" href="sql-statements.html#show-warnings" title="13.7.7.40 SHOW WARNINGS Statement"><code class="literal">SHOW
              WARNINGS</code></a> indicates which hints were used.
              Ignored hints are not displayed.
</p></li></ul>
</div>
<p>
          A hint comment may contain multiple hints, but a query block
          cannot contain multiple hint comments. This is valid:
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ BNL(t1) BKA(t2) */ ...</pre><p>
          But this is invalid:
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ BNL(t1) */ /* BKA(t2) */ ...</pre><p>
          When a hint comment contains multiple hints, the possibility
          of duplicates and conflicts exists. The following general
          guidelines apply. For specific hint types, additional rules
          may apply, as indicated in the hint descriptions.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Duplicate hints: For a hint such as <code class="literal">/*+ MRR(idx1)
              MRR(idx1) */</code>, MySQL uses the first hint and
              issues a warning about the duplicate hint.
            </p></li><li class="listitem"><p>
              Conflicting hints: For a hint such as <code class="literal">/*+
              MRR(idx1) NO_MRR(idx1) */</code>, MySQL uses the first
              hint and issues a warning about the second conflicting
              hint.
</p></li></ul>
</div>
<p>
          Query block names are identifiers and follow the usual rules
          about what names are valid and how to quote them (see
          <a class="xref" href="language-structure.html#identifiers" title="9.2 Schema Object Names">Section 9.2, “Schema Object Names”</a>).
        </p><p>
          Hint names, query block names, and strategy names are not case
          sensitive. References to table and index names follow the
          usual identifier case sensitivity rules (see
          <a class="xref" href="language-structure.html#identifier-case-sensitivity" title="9.2.3 Identifier Case Sensitivity">Section 9.2.3, “Identifier Case Sensitivity”</a>).
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="optimizer-hints-join-order"></a>Join-Order Optimizer Hints</h4>

</div>

</div>

</div>
<p>
          Join-order hints affect the order in which the optimizer joins
          tables.
        </p><p>
          Syntax of the
          <a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_FIXED_ORDER</code></a> hint:
        </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>hint_name</code></em>([@<em class="replaceable"><code>query_block_name</code></em>])
</pre><p>
          Syntax of other join-order hints:
        </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>hint_name</code></em>([@<em class="replaceable"><code>query_block_name</code></em>] <em class="replaceable"><code>tbl_name</code></em> [, <em class="replaceable"><code>tbl_name</code></em>] ...)
<em class="replaceable"><code>hint_name</code></em>(<em class="replaceable"><code>tbl_name</code></em>[@<em class="replaceable"><code>query_block_name</code></em>] [, <em class="replaceable"><code>tbl_name</code></em>[@<em class="replaceable"><code>query_block_name</code></em>]] ...)
</pre><p>
          The syntax refers to these terms:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <em class="replaceable"><code>hint_name</code></em>: These hint names are
              permitted:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                  <a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_FIXED_ORDER</code></a>:
                  Force the optimizer to join tables using the order in
                  which they appear in the <code class="literal">FROM</code>
                  clause. This is the same as specifying <code class="literal">SELECT
                  STRAIGHT_JOIN</code>.
                </p></li><li class="listitem"><p>
                  <a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_ORDER</code></a>: Instruct
                  the optimizer to join tables using the specified table
                  order. The hint applies to the named tables. The
                  optimizer may place tables that are not named anywhere
                  in the join order, including between specified tables.
                </p></li><li class="listitem"><p>
                  <a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_PREFIX</code></a>:
                  Instruct the optimizer to join tables using the
                  specified table order for the first tables of the join
                  execution plan. The hint applies to the named tables.
                  The optimizer places all other tables after the named
                  tables.
                </p></li><li class="listitem"><p>
                  <a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_SUFFIX</code></a>:
                  Instruct the optimizer to join tables using the
                  specified table order for the last tables of the join
                  execution plan. The hint applies to the named tables.
                  The optimizer places all other tables before the named
                  tables.
</p></li></ul>
</div>
</li><li class="listitem"><p>
              <em class="replaceable"><code>tbl_name</code></em>: The name of a table
              used in the statement. A hint that names tables applies to
              all tables that it names. The
              <a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_FIXED_ORDER</code></a> hint
              names no tables and applies to all tables in the
              <code class="literal">FROM</code> clause of the query block in which
              it occurs.
            </p><p>
              If a table has an alias, hints must refer to the alias,
              not the table name.
            </p><p>
              Table names in hints cannot be qualified with schema
              names.
            </p></li><li class="listitem"><p>
              <em class="replaceable"><code>query_block_name</code></em>: The query
              block to which the hint applies. If the hint includes no
              leading
              <code class="literal">@<em class="replaceable"><code>query_block_name</code></em></code>,
              the hint applies to the query block in which it occurs.
              For
              <code class="literal"><em class="replaceable"><code>tbl_name</code></em>@<em class="replaceable"><code>query_block_name</code></em></code>
              syntax, the hint applies to the named table in the named
              query block. To assign a name to a query block, see
              <a class="xref" href="optimization.html#optimizer-hints-query-block-naming" title="Optimizer Hints for Naming Query Blocks">Optimizer Hints for Naming Query Blocks</a>.
</p></li></ul>
</div>
<p>
          Example:
        </p><pre data-lang="sql" class="programlisting">SELECT
/*+ JOIN_PREFIX(t2, t5@subq2, t4@subq1)
    JOIN_ORDER(t4@subq1, t3)
    JOIN_SUFFIX(t1) */
COUNT(*) FROM t1 JOIN t2 JOIN t3
           WHERE t1.f1 IN (SELECT /*+ QB_NAME(subq1) */ f1 FROM t4)
             AND t2.f1 IN (SELECT /*+ QB_NAME(subq2) */ f1 FROM t5);</pre><p>
          Hints control the behavior of semijoin tables that are merged
          to the outer query block. If subqueries
          <code class="literal">subq1</code> and <code class="literal">subq2</code> are
          converted to semijoins, tables <code class="literal">t4@subq1</code> and
          <code class="literal">t5@subq2</code> are merged to the outer query
          block. In this case, the hint specified in the outer query
          block controls the behavior of <code class="literal">t4@subq1</code>,
          <code class="literal">t5@subq2</code> tables.
        </p><p>
          The optimizer resolves join-order hints according to these
          principles:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Multiple hint instances
            </p><p>
              Only one <a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_PREFIX</code></a> and
              <a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_SUFFIX</code></a> hint of each
              type are applied. Any later hints of the same type are
              ignored with a warning.
              <a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_ORDER</code></a> can be
              specified several times.
            </p><p>
              Examples:
            </p><pre data-lang="sql" class="programlisting">/*+ JOIN_PREFIX(t1) JOIN_PREFIX(t2) */</pre><p>
              The second <a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_PREFIX</code></a>
              hint is ignored with a warning.
            </p><pre data-lang="sql" class="programlisting">/*+ JOIN_PREFIX(t1) JOIN_SUFFIX(t2) */</pre><p>
              Both hints are applicable. No warning occurs.
            </p><pre data-lang="sql" class="programlisting">/*+ JOIN_ORDER(t1, t2) JOIN_ORDER(t2, t3) */</pre><p>
              Both hints are applicable. No warning occurs.
            </p></li><li class="listitem"><p>
              Conflicting hints
            </p><p>
              In some cases hints can conflict, such as when
              <a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_ORDER</code></a> and
              <a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_PREFIX</code></a> have table
              orders that are impossible to apply at the same time:
            </p><pre data-lang="sql" class="programlisting">SELECT /*+ JOIN_ORDER(t1, t2) JOIN_PREFIX(t2, t1) */ ... FROM t1, t2;</pre><p>
              In this case, the first specified hint is applied and
              subsequent conflicting hints are ignored with no warning.
              A valid hint that is impossible to apply is silently
              ignored with no warning.
            </p></li><li class="listitem"><p>
              Ignored hints
            </p><p>
              A hint is ignored if a table specified in the hint has a
              circular dependency.
            </p><p>
              Example:
            </p><pre data-lang="sql" class="programlisting">/*+ JOIN_ORDER(t1, t2) JOIN_PREFIX(t2, t1) */</pre><p>
              The <a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_ORDER</code></a> hint sets
              table <code class="literal">t2</code> dependent on
              <code class="literal">t1</code>. The
              <a class="link" href="optimization.html#optimizer-hints-join-order" title="Join-Order Optimizer Hints"><code class="literal">JOIN_PREFIX</code></a> hint is
              ignored because table <code class="literal">t1</code> cannot be
              dependent on <code class="literal">t2</code>. Ignored hints are not
              displayed in extended
              <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output.
            </p></li><li class="listitem"><p>
              Interaction with <a class="link" href="optimization.html#jointype_const"><code class="literal">const</code></a>
              tables
            </p><p>
              The MySQL optimizer places <code class="literal">const</code> tables
              first in the join order, and the position of a
              <code class="literal">const</code> table cannot be affected by
              hints. References to <code class="literal">const</code> tables in
              join-order hints are ignored, although the hint is still
              applicable. For example, these are equivalent:
            </p><pre data-lang="sql" class="programlisting">JOIN_ORDER(t1, <em class="replaceable"><code>const_tbl</code></em>, t2)
JOIN_ORDER(t1, t2)
</pre><p>
              Accepted hints shown in extended
              <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> output include
              <code class="literal">const</code> tables as they were specified.
            </p></li><li class="listitem"><p>
              Interaction with types of join operations
            </p><p>
              MySQL supports several type of joins:
              <code class="literal">LEFT</code>, <code class="literal">RIGHT</code>,
              <code class="literal">INNER</code>, <code class="literal">CROSS</code>,
              <code class="literal">STRAIGHT_JOIN</code>. A hint that conflicts
              with the specified type of join is ignored with no
              warning.
            </p><p>
              Example:
            </p><pre data-lang="sql" class="programlisting">SELECT /*+ JOIN_PREFIX(t1, t2) */FROM t2 LEFT JOIN t1;</pre><p>
              Here a conflict occurs between the requested join order in
              the hint and the order required by the <code class="literal">LEFT
              JOIN</code>. The hint is ignored with no warning.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="optimizer-hints-table-level"></a>Table-Level Optimizer Hints</h4>

</div>

</div>

</div>
<p>
          Table-level hints affect:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Use of the Block Nested-Loop (BNL) and Batched Key Access
              (BKA) join-processing algorithms (see
              <a class="xref" href="optimization.html#bnl-bka-optimization" title="8.2.1.12 Block Nested-Loop and Batched Key Access Joins">Section 8.2.1.12, “Block Nested-Loop and Batched Key Access Joins”</a>).
            </p></li><li class="listitem"><p>
              Whether derived tables, view references, or common table
              expressions should be merged into the outer query block,
              or materialized using an internal temporary table.
</p></li></ul>
</div>
<p>
          These hint types apply to specific tables, or all tables in a
          query block.
        </p><p>
          Syntax of table-level hints:
        </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>hint_name</code></em>([@<em class="replaceable"><code>query_block_name</code></em>] [<em class="replaceable"><code>tbl_name</code></em> [, <em class="replaceable"><code>tbl_name</code></em>] ...])
<em class="replaceable"><code>hint_name</code></em>([<em class="replaceable"><code>tbl_name</code></em>@<em class="replaceable"><code>query_block_name</code></em> [, <em class="replaceable"><code>tbl_name</code></em>@<em class="replaceable"><code>query_block_name</code></em>] ...])
</pre><p>
          The syntax refers to these terms:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <em class="replaceable"><code>hint_name</code></em>: These hint names are
              permitted:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                  <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">BKA</code></a>,
                  <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">NO_BKA</code></a>: Enable or
                  disable BKA for the specified tables.
                </p></li><li class="listitem"><p>
                  <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">BNL</code></a>,
                  <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">NO_BNL</code></a>: Enable or
                  disable BNL for the specified tables.
                </p></li><li class="listitem"><p>
                  <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">HASH_JOIN</code></a>,
                  <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">NO_HASH_JOIN</code></a>: Enable
                  or disable use of a hash join for the specified tables
                  (MySQL 8.0.18 only; has no effect in MySQL 8.0.19 or
                  later).
                </p></li><li class="listitem"><p>
                  <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">MERGE</code></a>,
                  <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">NO_MERGE</code></a>: Enable
                  merging for the specified tables, view references or
                  common table expressions; or disable merging and use
                  materialization instead.
</p><a class="indexterm" name="idm46444347539808"></a><a class="indexterm" name="idm46444347538320"></a><a class="indexterm" name="idm46444347536832"></a></li></ul>
</div>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">
<div class="admon-title">
Note
</div>
<p>
                To use a BNL or BKA hint to enable join buffering for
                any inner table of an outer join, join buffering must be
                enabled for all inner tables of the outer join.
</p>
</div>
</li><li class="listitem"><p>
              <em class="replaceable"><code>tbl_name</code></em>: The name of a table
              used in the statement. The hint applies to all tables that
              it names. If the hint names no tables, it applies to all
              tables of the query block in which it occurs.
            </p><p>
              If a table has an alias, hints must refer to the alias,
              not the table name.
            </p><p>
              Table names in hints cannot be qualified with schema
              names.
            </p></li><li class="listitem"><p>
              <em class="replaceable"><code>query_block_name</code></em>: The query
              block to which the hint applies. If the hint includes no
              leading
              <code class="literal">@<em class="replaceable"><code>query_block_name</code></em></code>,
              the hint applies to the query block in which it occurs.
              For
              <code class="literal"><em class="replaceable"><code>tbl_name</code></em>@<em class="replaceable"><code>query_block_name</code></em></code>
              syntax, the hint applies to the named table in the named
              query block. To assign a name to a query block, see
              <a class="xref" href="optimization.html#optimizer-hints-query-block-naming" title="Optimizer Hints for Naming Query Blocks">Optimizer Hints for Naming Query Blocks</a>.
</p></li></ul>
</div>
<p>
          Examples:
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ NO_BKA(t1, t2) */ t1.* FROM t1 INNER JOIN t2 INNER JOIN t3;
SELECT /*+ NO_BNL() BKA(t1) */ t1.* FROM t1 INNER JOIN t2 INNER JOIN t3;
SELECT /*+ NO_MERGE(dt) */ * FROM (SELECT * FROM t1) AS dt;</pre><p>
          A table-level hint applies to tables that receive records from
          previous tables, not sender tables. Consider this statement:
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ BNL(t2) */ FROM t1, t2;</pre><p>
          If the optimizer chooses to process <code class="literal">t1</code>
          first, it applies a Block Nested-Loop join to
          <code class="literal">t2</code> by buffering the rows from
          <code class="literal">t1</code> before starting to read from
          <code class="literal">t2</code>. If the optimizer instead chooses to
          process <code class="literal">t2</code> first, the hint has no effect
          because <code class="literal">t2</code> is a sender table.
        </p><p>
          For the <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">MERGE</code></a> and
          <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">NO_MERGE</code></a> hints, these
          precedence rules apply:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              A hint takes precedence over any optimizer heuristic that
              is not a technical constraint. (If providing a hint as a
              suggestion has no effect, the optimizer has a reason for
              ignoring it.)
            </p></li><li class="listitem"><p>
              A hint takes precedence over the
              <code class="literal">derived_merge</code> flag of the
              <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
              variable.
            </p></li><li class="listitem"><p>
              For view references, an
              <code class="literal">ALGORITHM={MERGE|TEMPTABLE}</code> clause in
              the view definition takes precedence over a hint specified
              in the query referencing the view.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="optimizer-hints-index-level"></a>Index-Level Optimizer Hints</h4>

</div>

</div>

</div>
<p>
          Index-level hints affect which index-processing strategies the
          optimizer uses for particular tables or indexes. These hint
          types affect use of Index Condition Pushdown (ICP),
          Multi-Range Read (MRR), Index Merge, and range optimizations
          (see <a class="xref" href="optimization.html#select-optimization" title="8.2.1 Optimizing SELECT Statements">Section 8.2.1, “Optimizing SELECT Statements”</a>).
        </p><p>
          Syntax of index-level hints:
        </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>hint_name</code></em>([@<em class="replaceable"><code>query_block_name</code></em>] <em class="replaceable"><code>tbl_name</code></em> [<em class="replaceable"><code>index_name</code></em> [, <em class="replaceable"><code>index_name</code></em>] ...])
<em class="replaceable"><code>hint_name</code></em>(<em class="replaceable"><code>tbl_name</code></em>@<em class="replaceable"><code>query_block_name</code></em> [<em class="replaceable"><code>index_name</code></em> [, <em class="replaceable"><code>index_name</code></em>] ...])
</pre><p>
          The syntax refers to these terms:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <em class="replaceable"><code>hint_name</code></em>: These hint names are
              permitted:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                  <a class="indexterm" name="idm46444347499312"></a>

                  <a class="indexterm" name="idm46444347498240"></a>

                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">GROUP_INDEX</code></a>,
                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_GROUP_INDEX</code></a>:
                  Enable or disable the specified index or indexes for
                  index scans for <code class="literal">GROUP BY</code>
                  operations. Equivalent to the index hints
                  <code class="literal">FORCE INDEX FOR GROUP BY</code>,
                  <code class="literal">IGNORE INDEX FOR GROUP BY</code>.
                  Available in MySQL 8.0.20 and later.
                </p></li><li class="listitem"><p>
                  <a class="indexterm" name="idm46444347491456"></a>

                  <a class="indexterm" name="idm46444347490384"></a>

                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">INDEX</code></a>,
                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_INDEX</code></a>: Acts as
                  the combination of
                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">JOIN_INDEX</code></a>,
                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">GROUP_INDEX</code></a>, and
                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">ORDER_INDEX</code></a>, forcing
                  the server to use the specified index or indexes for
                  any and all scopes, or as the combination of
                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_JOIN_INDEX</code></a>,
                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_GROUP_INDEX</code></a>, and
                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_ORDER_INDEX</code></a>,
                  which causes the server to ignore the specified index
                  or indexes for any and all scopes. Equivalent to
                  <code class="literal">FORCE INDEX</code>, <code class="literal">IGNORE
                  INDEX</code>. Available beginning with MySQL
                  8.0.20.
                </p></li><li class="listitem"><p>
                  <a class="indexterm" name="idm46444347476576"></a>

                  <a class="indexterm" name="idm46444347475504"></a>

                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">INDEX_MERGE</code></a>,
                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_INDEX_MERGE</code></a>:
                  Enable or disable the Index Merge access method for
                  the specified table or indexes. For information about
                  this access method, see
                  <a class="xref" href="optimization.html#index-merge-optimization" title="8.2.1.3 Index Merge Optimization">Section 8.2.1.3, “Index Merge Optimization”</a>. These
                  hints apply to all three Index Merge algorithms.
                </p><p>
                  The <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">INDEX_MERGE</code></a> hint
                  forces the optimizer to use Index Merge for the
                  specified table using the specified set of indexes. If
                  no index is specified, the optimizer considers all
                  possible index combinations and selects the least
                  expensive one. The hint may be ignored if the index
                  combination is inapplicable to the given statement.
                </p><p>
                  The <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_INDEX_MERGE</code></a>
                  hint disables Index Merge combinations that involve
                  any of the specified indexes. If the hint specifies no
                  indexes, Index Merge is not permitted for the table.
                </p></li><li class="listitem"><p>
                  <a class="indexterm" name="idm46444347466128"></a>

                  <a class="indexterm" name="idm46444347465056"></a>

                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">JOIN_INDEX</code></a>,
                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_JOIN_INDEX</code></a>:
                  Forces MySQL to use or ignore the specified index or
                  indexes for any access method, such as
                  <code class="literal">ref</code>, <code class="literal">range</code>,
                  <code class="literal">index_merge</code>, and so on. Equivalent
                  to <code class="literal">FORCE INDEX FOR JOIN</code>,
                  <code class="literal">IGNORE INDEX FOR JOIN</code>. Available in
                  MySQL 8.0.20 and later.
                </p></li><li class="listitem"><p>
                  <a class="indexterm" name="idm46444347456928"></a>

                  <a class="indexterm" name="idm46444347455888"></a>

                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">MRR</code></a>,
                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_MRR</code></a>: Enable or
                  disable MRR for the specified table or indexes. MRR
                  hints apply only to <code class="literal">InnoDB</code> and
                  <code class="literal">MyISAM</code> tables. For information
                  about this access method, see
                  <a class="xref" href="optimization.html#mrr-optimization" title="8.2.1.11 Multi-Range Read Optimization">Section 8.2.1.11, “Multi-Range Read Optimization”</a>.
                </p></li><li class="listitem"><p>
                  <a class="indexterm" name="idm46444347449296"></a>

                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_ICP</code></a>: Disable ICP
                  for the specified table or indexes. By default, ICP is
                  a candidate optimization strategy, so there is no hint
                  for enabling it. For information about this access
                  method, see
                  <a class="xref" href="optimization.html#index-condition-pushdown-optimization" title="8.2.1.6 Index Condition Pushdown Optimization">Section 8.2.1.6, “Index Condition Pushdown Optimization”</a>.
                </p></li><li class="listitem"><p>
                  <a class="indexterm" name="idm46444347445184"></a>

                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_RANGE_OPTIMIZATION</code></a>:
                  Disable index range access for the specified table or
                  indexes. This hint also disables Index Merge and Loose
                  Index Scan for the table or indexes. By default, range
                  access is a candidate optimization strategy, so there
                  is no hint for enabling it.
                </p><p>
                  This hint may be useful when the number of ranges may
                  be high and range optimization would require many
                  resources.
                </p></li><li class="listitem"><p>
                  <a class="indexterm" name="idm46444347441152"></a>

                  <a class="indexterm" name="idm46444347440080"></a>

                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">ORDER_INDEX</code></a>,
                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_ORDER_INDEX</code></a>:
                  Cause MySQL to use or to ignore the specified index or
                  indexes for sorting rows. Equivalent to <code class="literal">FORCE
                  INDEX FOR ORDER BY</code>, <code class="literal">IGNORE INDEX
                  FOR ORDER BY</code>. Available beginning with MySQL
                  8.0.20.
                </p></li><li class="listitem"><p>
                  <a class="indexterm" name="idm46444347434064"></a>

                  <a class="indexterm" name="idm46444347432992"></a>

                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">SKIP_SCAN</code></a>,
                  <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_SKIP_SCAN</code></a>: Enable
                  or disable the Skip Scan access method for the
                  specified table or indexes. For information about this
                  access method, see
                  <a class="xref" href="optimization.html#range-access-skip-scan" title="Skip Scan Range Access Method">Skip Scan Range Access Method</a>. These hints
                  are available as of MySQL 8.0.13.
                </p><p>
                  The <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">SKIP_SCAN</code></a> hint
                  forces the optimizer to use Skip Scan for the
                  specified table using the specified set of indexes. If
                  no index is specified, the optimizer considers all
                  possible indexes and selects the least expensive one.
                  The hint may be ignored if the index is inapplicable
                  to the given statement.
                </p><p>
                  The <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_SKIP_SCAN</code></a>
                  hint disables Skip Scan for the specified indexes. If
                  the hint specifies no indexes, Skip Scan is not
                  permitted for the table.
</p></li></ul>
</div>
</li><li class="listitem"><p>
              <em class="replaceable"><code>tbl_name</code></em>: The table to which
              the hint applies.
            </p></li><li class="listitem"><p>
              <em class="replaceable"><code>index_name</code></em>: The name of an
              index in the named table. The hint applies to all indexes
              that it names. If the hint names no indexes, it applies to
              all indexes in the table.
            </p><p>
              To refer to a primary key, use the name
              <code class="literal">PRIMARY</code>. To see the index names for a
              table, use <a class="link" href="sql-statements.html#show-index" title="13.7.7.22 SHOW INDEX Statement"><code class="literal">SHOW INDEX</code></a>.
            </p></li><li class="listitem"><p>
              <em class="replaceable"><code>query_block_name</code></em>: The query
              block to which the hint applies. If the hint includes no
              leading
              <code class="literal">@<em class="replaceable"><code>query_block_name</code></em></code>,
              the hint applies to the query block in which it occurs.
              For
              <code class="literal"><em class="replaceable"><code>tbl_name</code></em>@<em class="replaceable"><code>query_block_name</code></em></code>
              syntax, the hint applies to the named table in the named
              query block. To assign a name to a query block, see
              <a class="xref" href="optimization.html#optimizer-hints-query-block-naming" title="Optimizer Hints for Naming Query Blocks">Optimizer Hints for Naming Query Blocks</a>.
</p></li></ul>
</div>
<p>
          Examples:
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ INDEX_MERGE(t1 f3, PRIMARY) */ f2 FROM t1
  WHERE f1 = 'o' AND f2 = f3 AND f3 &lt;= 4;
SELECT /*+ MRR(t1) */ * FROM t1 WHERE f2 &lt;= 3 AND 3 &lt;= f3;
SELECT /*+ NO_RANGE_OPTIMIZATION(t3 PRIMARY, f2_idx) */ f1
  FROM t3 WHERE f1 &gt; 30 AND f1 &lt; 33;
INSERT INTO t3(f1, f2, f3)
  (SELECT /*+ NO_ICP(t2) */ t2.f1, t2.f2, t2.f3 FROM t1,t2
   WHERE t1.f1=t2.f1 AND t2.f2 BETWEEN t1.f1
   AND t1.f2 AND t2.f2 + 1 &gt;= t1.f1 + 1);
SELECT /*+ SKIP_SCAN(t1 PRIMARY) */ f1, f2
  FROM t1 WHERE f2 &gt; 40;</pre><p>
          The following examples use the Index Merge hints, but other
          index-level hints follow the same principles regarding hint
          ignoring and precedence of optimizer hints in relation to the
          <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
          variable or index hints.
        </p><p>
          Assume that table <code class="literal">t1</code> has columns
          <code class="literal">a</code>, <code class="literal">b</code>,
          <code class="literal">c</code>, and <code class="literal">d</code>; and that
          indexes named <code class="literal">i_a</code>, <code class="literal">i_b</code>,
          and <code class="literal">i_c</code> exist on <code class="literal">a</code>,
          <code class="literal">b</code>, and <code class="literal">c</code>, respectively:
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ INDEX_MERGE(t1 i_a, i_b, i_c)*/ * FROM t1
  WHERE a = 1 AND b = 2 AND c = 3 AND d = 4;</pre><p>
          Index Merge is used for <code class="literal">(i_a, i_b, i_c)</code> in
          this case.
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ INDEX_MERGE(t1 i_a, i_b, i_c)*/ * FROM t1
  WHERE b = 1 AND c = 2 AND d = 3;</pre><p>
          Index Merge is used for <code class="literal">(i_b, i_c)</code> in this
          case.
        </p><pre data-lang="sql" class="programlisting">/*+ INDEX_MERGE(t1 i_a, i_b) NO_INDEX_MERGE(t1 i_b) */</pre><p>
          <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_INDEX_MERGE</code></a> is ignored
          because there is a preceding hint for the same table.
        </p><pre data-lang="sql" class="programlisting">/*+ NO_INDEX_MERGE(t1 i_a, i_b) INDEX_MERGE(t1 i_b) */</pre><p>
          <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">INDEX_MERGE</code></a> is ignored
          because there is a preceding hint for the same table.
        </p><p>
          For the <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">INDEX_MERGE</code></a> and
          <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_INDEX_MERGE</code></a> optimizer
          hints, these precedence rules apply:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              If an optimizer hint is specified and is applicable, it
              takes precedence over the Index Merge-related flags of the
              <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
              variable.
            </p><pre data-lang="sql" class="programlisting">SET optimizer_switch='index_merge_intersection=off';
SELECT /*+ INDEX_MERGE(t1 i_b, i_c) */ * FROM t1
WHERE b = 1 AND c = 2 AND d = 3;</pre><p>
              The hint takes precedence over
              <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a>. Index
              Merge is used for <code class="literal">(i_b, i_c)</code> in this
              case.
            </p><pre data-lang="sql" class="programlisting">SET optimizer_switch='index_merge_intersection=on';
SELECT /*+ INDEX_MERGE(t1 i_b) */ * FROM t1
WHERE b = 1 AND c = 2 AND d = 3;</pre><p>
              The hint specifies only one index, so it is inapplicable,
              and the <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a>
              flag (<code class="literal">on</code>) applies. Index Merge is used
              if the optimizer assesses it to be cost efficient.
            </p><pre data-lang="sql" class="programlisting">SET optimizer_switch='index_merge_intersection=off';
SELECT /*+ INDEX_MERGE(t1 i_b) */ * FROM t1
WHERE b = 1 AND c = 2 AND d = 3;</pre><p>
              The hint specifies only one index, so it is inapplicable,
              and the <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a>
              flag (<code class="literal">off</code>) applies. Index Merge is not
              used.
            </p></li><li class="listitem"><p>
              The index-level optimizer hints
              <code class="literal">GROUP_INDEX</code>, <code class="literal">INDEX</code>,
              <code class="literal">JOIN_INDEX</code>, and
              <code class="literal">ORDER_INDEX</code> all take precedence over
              the equivalent <code class="literal">FORCE INDEX</code> hints; that
              is, they cause the <code class="literal">FORCE INDEX</code> hints to
              be ignored. Likewise, the
              <code class="literal">NO_GROUP_INDEX</code>,
              <code class="literal">NO_INDEX</code>,
              <code class="literal">NO_JOIN_INDEX</code>, and
              <code class="literal">NO_ORDER_INDEX</code> hints all take
              precedence over any <code class="literal">IGNORE INDEX</code>
              equivalents, also causing them to be ignored.
            </p><p>
              The index-level optimizer hints
              <code class="literal">GROUP_INDEX</code>,
              <code class="literal">NO_GROUP_INDEX</code>,
              <code class="literal">INDEX</code>,<code class="literal">NO_INDEX</code>,
              <code class="literal">JOIN_INDEX</code>,<code class="literal">NO_JOIN_INDEX</code>,
              <code class="literal">ORDER_INDEX</code>, and
              <code class="literal">NO_ORDER_INDEX</code> hints all take
              precedence over all other optimizer hints, including other
              index-level optimizer hints. Any other optimizer hints are
              applied only to the indexes permitted by these.
            </p><p>
              The <code class="literal">GROUP_INDEX</code>,
              <code class="literal">INDEX</code>, <code class="literal">JOIN_INDEX</code>,
              and <code class="literal">ORDER_INDEX</code> hints are all
              equivalent to <code class="literal">FORCE INDEX</code> and not to
              <code class="literal">USE INDEX</code>. This is because using one or
              more of these hints means that a table scan is used only
              if there is no way to use one of the named indexes to find
              rows in the table. To cause MySQL to use the same index or
              set of indexes as with a given instance of <code class="literal">USE
              INDEX</code>, you can use <code class="literal">NO_INDEX</code>,
              <code class="literal">NO_JOIN_INDEX</code>,
              <code class="literal">NO_GROUP_INDEX</code>,
              <code class="literal">NO_ORDER_INDEX</code>, or some combination of
              these.
            </p><p>
              To replicate the effect that <code class="literal">USE INDEX</code>
              has in the query <code class="literal">SELECT a,c FROM t1 USE INDEX FOR
              ORDER BY (i_a) ORDER BY a</code>, you can use the
              <code class="literal">NO_ORDER_INDEX</code> optimizer hint to cover
              all indexes on the table except the one that is desired
              like this:
            </p><pre data-lang="sql" class="programlisting">SELECT /*+ NO_ORDER_INDEX(t1 i_b,i_c) */ a,c
    FROM t1
    ORDER BY a;</pre><p>
              Attempting to combine <code class="literal">NO_ORDER_INDEX</code>
              for the table as a whole with <code class="literal">USE INDEX FOR ORDER
              BY</code> does not work to do this, because
              <code class="literal">NO_ORDER_BY</code> causes <code class="literal">USE
              INDEX</code> to be ignored, as shown here:
            </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>EXPLAIN SELECT /*+ NO_ORDER_INDEX(t1) */ a,c FROM t1</code></strong>
    -&gt;     <strong class="userinput"><code>USE INDEX FOR ORDER BY (i_a) ORDER BY a\G</code></strong>
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: t1
   partitions: NULL
         type: ALL
possible_keys: NULL
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 256
     filtered: 100.00
        Extra: Using filesort
</pre></li><li class="listitem"><p>
              The <code class="literal">USE INDEX</code>, <code class="literal">FORCE
              INDEX</code>, and <code class="literal">IGNORE INDEX</code> index
              hints have higher priority than the
              <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">INDEX_MERGE</code></a> and
              <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_INDEX_MERGE</code></a> optimizer
              hints.
            </p><pre data-lang="sql" class="programlisting">/*+ INDEX_MERGE(t1 i_a, i_b, i_c) */ ... IGNORE INDEX i_a</pre><p>
              <code class="literal">IGNORE INDEX</code> takes precedence over
              <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">INDEX_MERGE</code></a>, so index
              <code class="literal">i_a</code> is excluded from the possible
              ranges for Index Merge.
            </p><pre data-lang="sql" class="programlisting">/*+ NO_INDEX_MERGE(t1 i_a, i_b) */ ... FORCE INDEX i_a, i_b</pre><p>
              Index Merge is disallowed for <code class="literal">i_a, i_b</code>
              because of <code class="literal">FORCE INDEX</code>, but the
              optimizer is forced to use either <code class="literal">i_a</code>
              or <code class="literal">i_b</code> for
              <a class="link" href="optimization.html#jointype_range"><code class="literal">range</code></a> or
              <a class="link" href="optimization.html#jointype_ref"><code class="literal">ref</code></a> access. There are
              no conflicts; both hints are applicable.
            </p></li><li class="listitem"><p>
              If an <code class="literal">IGNORE INDEX</code> hint names multiple
              indexes, those indexes are unavailable for Index Merge.
            </p></li><li class="listitem"><p>
              The <code class="literal">FORCE INDEX</code> and <code class="literal">USE
              INDEX</code> hints make only the named indexes to be
              available for Index Merge.
            </p><pre data-lang="sql" class="programlisting">SELECT /*+ INDEX_MERGE(t1 i_a, i_b, i_c) */ a FROM t1
FORCE INDEX (i_a, i_b) WHERE c = 'h' AND a = 2 AND b = 'b';</pre><p>
              The Index Merge intersection access algorithm is used for
              <code class="literal">(i_a, i_b)</code>. The same is true if
              <code class="literal">FORCE INDEX</code> is changed to <code class="literal">USE
              INDEX</code>.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="optimizer-hints-subquery"></a>Subquery Optimizer Hints</h4>

</div>

</div>

</div>
<p>
          Subquery hints affect whether to use semijoin transformations
          and which semijoin strategies to permit, and, when semijoins
          are not used, whether to use subquery materialization or
          <code class="literal">IN</code>-to-<code class="literal">EXISTS</code>
          transformations. For more information about these
          optimizations, see <a class="xref" href="optimization.html#subquery-optimization" title="8.2.2 Optimizing Subqueries, Derived Tables, View References, and Common Table Expressions">Section 8.2.2, “Optimizing Subqueries, Derived Tables, View References, and Common Table
        Expressions”</a>.
        </p><p>
          Syntax of hints that affect semijoin strategies:
        </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>hint_name</code></em>([@<em class="replaceable"><code>query_block_name</code></em>] [<em class="replaceable"><code>strategy</code></em> [, <em class="replaceable"><code>strategy</code></em>] ...])
</pre><p>
          The syntax refers to these terms:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <em class="replaceable"><code>hint_name</code></em>: These hint names are
              permitted:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                  <a class="link" href="optimization.html#optimizer-hints-subquery" title="Subquery Optimizer Hints"><code class="literal">SEMIJOIN</code></a>,
                  <a class="link" href="optimization.html#optimizer-hints-subquery" title="Subquery Optimizer Hints"><code class="literal">NO_SEMIJOIN</code></a>: Enable
                  or disable the named semijoin strategies.
</p></li></ul>
</div>
</li><li class="listitem"><p>
              <em class="replaceable"><code>strategy</code></em>: A semijoin strategy
              to be enabled or disabled. These strategy names are
              permitted: <code class="literal">DUPSWEEDOUT</code>,
              <code class="literal">FIRSTMATCH</code>,
              <code class="literal">LOOSESCAN</code>,
              <code class="literal">MATERIALIZATION</code>.
            </p><p>
              For <a class="link" href="optimization.html#optimizer-hints-subquery" title="Subquery Optimizer Hints"><code class="literal">SEMIJOIN</code></a> hints, if
              no strategies are named, semijoin is used if possible
              based on the strategies enabled according to the
              <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
              variable. If strategies are named but inapplicable for the
              statement, <code class="literal">DUPSWEEDOUT</code> is used.
            </p><p>
              For <a class="link" href="optimization.html#optimizer-hints-subquery" title="Subquery Optimizer Hints"><code class="literal">NO_SEMIJOIN</code></a> hints,
              if no strategies are named, semijoin is not used. If
              strategies are named that rule out all applicable
              strategies for the statement,
              <code class="literal">DUPSWEEDOUT</code> is used.
</p></li></ul>
</div>
<p>
          If one subquery is nested within another and both are merged
          into a semijoin of an outer query, any specification of
          semijoin strategies for the innermost query are ignored.
          <a class="link" href="optimization.html#optimizer-hints-subquery" title="Subquery Optimizer Hints"><code class="literal">SEMIJOIN</code></a> and
          <a class="link" href="optimization.html#optimizer-hints-subquery" title="Subquery Optimizer Hints"><code class="literal">NO_SEMIJOIN</code></a> hints can still
          be used to enable or disable semijoin transformations for such
          nested subqueries.
        </p><p>
          If <code class="literal">DUPSWEEDOUT</code> is disabled, on occasion the
          optimizer may generate a query plan that is far from optimal.
          This occurs due to heuristic pruning during greedy search,
          which can be avoided by setting
          <a class="link" href="server-administration.html#sysvar_optimizer_prune_level"><code class="literal">optimizer_prune_level=0</code></a>.
        </p><p>
          Examples:
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ NO_SEMIJOIN(@subq1 FIRSTMATCH, LOOSESCAN) */ * FROM t2
  WHERE t2.a IN (SELECT /*+ QB_NAME(subq1) */ a FROM t3);
SELECT /*+ SEMIJOIN(@subq1 MATERIALIZATION, DUPSWEEDOUT) */ * FROM t2
  WHERE t2.a IN (SELECT /*+ QB_NAME(subq1) */ a FROM t3);</pre><p>
          Syntax of hints that affect whether to use subquery
          materialization or
          <code class="literal">IN</code>-to-<code class="literal">EXISTS</code>
          transformations:
        </p><pre data-lang="sql" class="programlisting">SUBQUERY([@<em class="replaceable"><code>query_block_name</code></em>] <em class="replaceable"><code>strategy</code></em>)
</pre><p>
          The hint name is always
          <a class="link" href="optimization.html#optimizer-hints-subquery" title="Subquery Optimizer Hints"><code class="literal">SUBQUERY</code></a>.
        </p><p>
          For <a class="link" href="optimization.html#optimizer-hints-subquery" title="Subquery Optimizer Hints"><code class="literal">SUBQUERY</code></a> hints, these
          <em class="replaceable"><code>strategy</code></em> values are permitted:
          <code class="literal">INTOEXISTS</code>,
          <code class="literal">MATERIALIZATION</code>.
        </p><p>
          Examples:
        </p><pre data-lang="sql" class="programlisting">SELECT id, a IN (SELECT /*+ SUBQUERY(MATERIALIZATION) */ a FROM t1) FROM t2;
SELECT * FROM t2 WHERE t2.a IN (SELECT /*+ SUBQUERY(INTOEXISTS) */ a FROM t1);</pre><p>
          For semijoin and <a class="link" href="optimization.html#optimizer-hints-subquery" title="Subquery Optimizer Hints"><code class="literal">SUBQUERY</code></a>
          hints, a leading
          <code class="literal">@<em class="replaceable"><code>query_block_name</code></em></code>
          specifies the query block to which the hint applies. If the
          hint includes no leading
          <code class="literal">@<em class="replaceable"><code>query_block_name</code></em></code>,
          the hint applies to the query block in which it occurs. To
          assign a name to a query block, see
          <a class="xref" href="optimization.html#optimizer-hints-query-block-naming" title="Optimizer Hints for Naming Query Blocks">Optimizer Hints for Naming Query Blocks</a>.
        </p><p>
          If a hint comment contains multiple subquery hints, the first
          is used. If there are other following hints of that type, they
          produce a warning. Following hints of other types are silently
          ignored.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="optimizer-hints-execution-time"></a>Statement Execution Time Optimizer Hints</h4>

</div>

</div>

</div>
<p>
          The <a class="link" href="optimization.html#optimizer-hints-execution-time" title="Statement Execution Time Optimizer Hints"><code class="literal">MAX_EXECUTION_TIME</code></a> hint
          is permitted only for <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>
          statements. It places a limit <em class="replaceable"><code>N</code></em> (a
          timeout value in milliseconds) on how long a statement is
          permitted to execute before the server terminates it:
        </p><pre data-lang="sql" class="programlisting">MAX_EXECUTION_TIME(<em class="replaceable"><code>N</code></em>)
</pre><p>
          Example with a timeout of 1 second (1000 milliseconds):
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ MAX_EXECUTION_TIME(1000) */ * FROM t1 INNER JOIN t2 WHERE ...</pre><p>
          The
          <a class="link" href="optimization.html#optimizer-hints-execution-time" title="Statement Execution Time Optimizer Hints"><code class="literal">MAX_EXECUTION_TIME(<em class="replaceable"><code>N</code></em>)</code></a>
          hint sets a statement execution timeout of
          <em class="replaceable"><code>N</code></em> milliseconds. If this option is
          absent or <em class="replaceable"><code>N</code></em> is 0, the statement
          timeout established by the
          <a class="link" href="server-administration.html#sysvar_max_execution_time"><code class="literal">max_execution_time</code></a> system
          variable applies.
        </p><p>
          The <a class="link" href="optimization.html#optimizer-hints-execution-time" title="Statement Execution Time Optimizer Hints"><code class="literal">MAX_EXECUTION_TIME</code></a> hint
          is applicable as follows:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              For statements with multiple <code class="literal">SELECT</code>
              keywords, such as unions or statements with subqueries,
              <a class="link" href="optimization.html#optimizer-hints-execution-time" title="Statement Execution Time Optimizer Hints"><code class="literal">MAX_EXECUTION_TIME</code></a>
              applies to the entire statement and must appear after the
              first <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>.
            </p></li><li class="listitem"><p>
              It applies to read-only
              <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements.
              Statements that are not read only are those that invoke a
              stored function that modifies data as a side effect.
            </p></li><li class="listitem"><p>
              It does not apply to <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>
              statements in stored programs and is ignored.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="optimizer-hints-set-var"></a>Variable-Setting Hint Syntax</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444347248128"></a><a class="indexterm" name="idm46444347247088"></a><a class="indexterm" name="idm46444347245600"></a><a class="indexterm" name="idm46444347244112"></a><p>
          The <a class="link" href="optimization.html#optimizer-hints-set-var" title="Variable-Setting Hint Syntax"><code class="literal">SET_VAR</code></a> hint sets the
          session value of a system variable temporarily (for the
          duration of a single statement). Examples:
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ SET_VAR(sort_buffer_size = 16M) */ name FROM people ORDER BY name;
INSERT /*+ SET_VAR(foreign_key_checks=OFF) */ INTO t2 VALUES(2);
SELECT /*+ SET_VAR(optimizer_switch = 'mrr_cost_based=off') */ 1;</pre><p>
          Syntax of the <a class="link" href="optimization.html#optimizer-hints-set-var" title="Variable-Setting Hint Syntax"><code class="literal">SET_VAR</code></a> hint:
        </p><pre data-lang="sql" class="programlisting">SET_VAR(<em class="replaceable"><code>var_name</code></em> = <em class="replaceable"><code>value</code></em>)
</pre><p>
          <em class="replaceable"><code>var_name</code></em> names a system variable
          that has a session value (although not all such variables can
          be named, as explained later).
          <em class="replaceable"><code>value</code></em> is the value to assign to the
          variable; the value must be a scalar.
        </p><p>
          <a class="link" href="optimization.html#optimizer-hints-set-var" title="Variable-Setting Hint Syntax"><code class="literal">SET_VAR</code></a> makes a temporary
          variable change, as demonstrated by these statements:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT @@unique_checks;</code></strong>
+-----------------+
| @@unique_checks |
+-----------------+
|               1 |
+-----------------+
mysql&gt; <strong class="userinput"><code>SELECT /*+ SET_VAR(unique_checks=OFF) */ @@unique_checks;</code></strong>
+-----------------+
| @@unique_checks |
+-----------------+
|               0 |
+-----------------+
mysql&gt; <strong class="userinput"><code>SELECT @@unique_checks;</code></strong>
+-----------------+
| @@unique_checks |
+-----------------+
|               1 |
+-----------------+
</pre><p>
          With <a class="link" href="optimization.html#optimizer-hints-set-var" title="Variable-Setting Hint Syntax"><code class="literal">SET_VAR</code></a>, there is no
          need to save and restore the variable value. This enables you
          to replace multiple statements by a single statement. Consider
          this sequence of statements:
        </p><pre data-lang="sql" class="programlisting">SET @saved_val = @@SESSION.<em class="replaceable"><code>var_name</code></em>;
SET @@SESSION.<em class="replaceable"><code>var_name</code></em> = <em class="replaceable"><code>value</code></em>;
SELECT ...
SET @@SESSION.<em class="replaceable"><code>var_name</code></em> = @saved_val;
</pre><p>
          The sequence can be replaced by this single statement:
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ SET_VAR(<em class="replaceable"><code>var_name</code></em> = <em class="replaceable"><code>value</code></em>) ...
</pre><p>
          Standalone
          <a class="link" href="sql-statements.html#set-variable" title="13.7.6.1 SET Syntax for Variable Assignment"><code class="literal">SET</code></a>
          statements permit any of these syntaxes for naming session
          variables:
        </p><pre data-lang="sql" class="programlisting">SET SESSION <em class="replaceable"><code>var_name</code></em> = <em class="replaceable"><code>value</code></em>;
SET @@SESSION.<em class="replaceable"><code>var_name</code></em> = <em class="replaceable"><code>value</code></em>;
SET @@.<em class="replaceable"><code>var_name</code></em> = <em class="replaceable"><code>value</code></em>;
</pre><p>
          Because the <a class="link" href="optimization.html#optimizer-hints-set-var" title="Variable-Setting Hint Syntax"><code class="literal">SET_VAR</code></a> hint
          applies only to session variables, session scope is implicit,
          and <code class="literal">SESSION</code>, <code class="literal">@@SESSION.</code>,
          and <code class="literal">@@</code> are neither needed nor permitted.
          Including explicit session-indicator syntax results in the
          <a class="link" href="optimization.html#optimizer-hints-set-var" title="Variable-Setting Hint Syntax"><code class="literal">SET_VAR</code></a> hint being ignored
          with a warning.
        </p><p>
          Not all session variables are permitted for use with
          <a class="link" href="optimization.html#optimizer-hints-set-var" title="Variable-Setting Hint Syntax"><code class="literal">SET_VAR</code></a>. Individual system
          variable descriptions indicate whether each variable is
          hintable; see <a class="xref" href="server-administration.html#server-system-variables" title="5.1.8 Server System Variables">Section 5.1.8, “Server System Variables”</a>. You
          can also check a system variable at runtime by attempting to
          use it with <a class="link" href="optimization.html#optimizer-hints-set-var" title="Variable-Setting Hint Syntax"><code class="literal">SET_VAR</code></a>. If the
          variable is not hintable, a warning occurs:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT /*+ SET_VAR(collation_server = 'utf8') */ 1;</code></strong>
+---+
| 1 |
+---+
| 1 |
+---+
1 row in set, 1 warning (0.00 sec)

mysql&gt; <strong class="userinput"><code>SHOW WARNINGS\G</code></strong>
*************************** 1. row ***************************
  Level: Warning
   Code: 4537
Message: Variable 'collation_server' cannot be set using SET_VAR hint.
</pre><p>
          <a class="link" href="optimization.html#optimizer-hints-set-var" title="Variable-Setting Hint Syntax"><code class="literal">SET_VAR</code></a> syntax permits
          setting only a single variable, but multiple hints can be
          given to set multiple variables:
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ SET_VAR(optimizer_switch = 'mrr_cost_based=off')
           SET_VAR(max_heap_table_size = 1G) */ 1;</pre><p>
          If several hints with the same variable name appear in the
          same statement, the first one is applied and the others are
          ignored with a warning:
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ SET_VAR(max_heap_table_size = 1G)
           SET_VAR(max_heap_table_size = 3G) */ 1;</pre><p>
          In this case, the second hint is ignored with a warning that
          it is conflicting.
        </p><p>
          A <a class="link" href="optimization.html#optimizer-hints-set-var" title="Variable-Setting Hint Syntax"><code class="literal">SET_VAR</code></a> hint is ignored
          with a warning if no system variable has the specified name or
          the variable value is incorrect:
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ SET_VAR(max_size = 1G) */ 1;
SELECT /*+ SET_VAR(optimizer_switch = 'mrr_cost_based=yes') */ 1;</pre><p>
          For the first statement, there is no
          <code class="literal">max_size</code> variable. For the second
          statement, <code class="literal">mrr_cost_flag</code> takes values of
          <code class="literal">on</code> or <code class="literal">off</code>, so attempting
          to set it to <code class="literal">yes</code> is incorrect. In each
          case, the hint is ignored with a warning.
        </p><p>
          The <a class="link" href="optimization.html#optimizer-hints-set-var" title="Variable-Setting Hint Syntax"><code class="literal">SET_VAR</code></a> hint is
          permitted only at the statement level. If used in a subquery,
          the hint is ignored with a warning.
        </p><p>
          Slave servers ignore <a class="link" href="optimization.html#optimizer-hints-set-var" title="Variable-Setting Hint Syntax"><code class="literal">SET_VAR</code></a>
          hints in replicated statements to avoid the potential for
          security issues.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="optimizer-hints-resource-group"></a>Resource Group Hint Syntax</h4>

</div>

</div>

</div>
<p>
          The <a class="link" href="optimization.html#optimizer-hints-resource-group" title="Resource Group Hint Syntax"><code class="literal">RESOURCE_GROUP</code></a> optimizer
          hint is used for resource group management (see
          <a class="xref" href="server-administration.html#resource-groups" title="5.1.15 Resource Groups">Section 5.1.15, “Resource Groups”</a>). This hint assigns the
          thread that executes a statement to the named resource group
          temporarily (for the duration of the statement). It requires
          the <a class="link" href="security.html#priv_resource-group-admin"><code class="literal">RESOURCE_GROUP_ADMIN</code></a> or
          <a class="link" href="security.html#priv_resource-group-user"><code class="literal">RESOURCE_GROUP_USER</code></a> privilege.
        </p><p>
          Examples:
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ RESOURCE_GROUP(USR_default) */ name FROM people ORDER BY name;
INSERT /*+ RESOURCE_GROUP(Batch) */ INTO t2 VALUES(2);</pre><p>
          Syntax of the <a class="link" href="optimization.html#optimizer-hints-resource-group" title="Resource Group Hint Syntax"><code class="literal">RESOURCE_GROUP</code></a>
          hint:
        </p><pre data-lang="sql" class="programlisting">RESOURCE_GROUP(<em class="replaceable"><code>group_name</code></em>)
</pre><p>
          <em class="replaceable"><code>group_name</code></em> indicates the resource
          group to which the thread should be assigned for the duration
          of statement execution. If the group is nonexistent, a warning
          occurs and the hint is ignored.
        </p><p>
          The <a class="link" href="optimization.html#optimizer-hints-resource-group" title="Resource Group Hint Syntax"><code class="literal">RESOURCE_GROUP</code></a> hint must
          appear after the initial statement keyword
          (<code class="literal">SELECT</code>, <code class="literal">INSERT</code>,
          <code class="literal">REPLACE</code>, <code class="literal">UPDATE</code>, or
          <code class="literal">DELETE</code>).
        </p><p>
          An alternative to
          <a class="link" href="optimization.html#optimizer-hints-resource-group" title="Resource Group Hint Syntax"><code class="literal">RESOURCE_GROUP</code></a> is the
          <a class="link" href="sql-statements.html#set-resource-group" title="13.7.2.4 SET RESOURCE GROUP Statement"><code class="literal">SET RESOURCE GROUP</code></a> statement,
          which nontemporarily assigns threads to a resource group. See
          <a class="xref" href="sql-statements.html#set-resource-group" title="13.7.2.4 SET RESOURCE GROUP Statement">Section 13.7.2.4, “SET RESOURCE GROUP Statement”</a>.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="optimizer-hints-query-block-naming"></a>Optimizer Hints for Naming Query Blocks</h4>

</div>

</div>

</div>
<p>
          Table-level, index-level, and subquery optimizer hints permit
          specific query blocks to be named as part of their argument
          syntax. To create these names, use the
          <a class="link" href="optimization.html#optimizer-hints-query-block-naming" title="Optimizer Hints for Naming Query Blocks"><code class="literal">QB_NAME</code></a> hint, which assigns
          a name to the query block in which it occurs:
        </p><pre data-lang="sql" class="programlisting">QB_NAME(<em class="replaceable"><code>name</code></em>)
</pre><p>
          <a class="link" href="optimization.html#optimizer-hints-query-block-naming" title="Optimizer Hints for Naming Query Blocks"><code class="literal">QB_NAME</code></a> hints can be used to
          make explicit in a clear way which query blocks other hints
          apply to. They also permit all non-query block name hints to
          be specified within a single hint comment for easier
          understanding of complex statements. Consider the following
          statement:
        </p><pre data-lang="sql" class="programlisting">SELECT ...
  FROM (SELECT ...
  FROM (SELECT ... FROM ...)) ...</pre><p>
          <a class="link" href="optimization.html#optimizer-hints-query-block-naming" title="Optimizer Hints for Naming Query Blocks"><code class="literal">QB_NAME</code></a> hints assign names
          to query blocks in the statement:
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ QB_NAME(qb1) */ ...
  FROM (SELECT /*+ QB_NAME(qb2) */ ...
  FROM (SELECT /*+ QB_NAME(qb3) */ ... FROM ...)) ...</pre><p>
          Then other hints can use those names to refer to the
          appropriate query blocks:
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ QB_NAME(qb1) MRR(@qb1 t1) BKA(@qb2) NO_MRR(@qb3t1 idx1, id2) */ ...
  FROM (SELECT /*+ QB_NAME(qb2) */ ...
  FROM (SELECT /*+ QB_NAME(qb3) */ ... FROM ...)) ...</pre><p>
          The resulting effect is as follows:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">MRR(@qb1 t1)</code></a> applies to
              table <code class="literal">t1</code> in query block
              <code class="literal">qb1</code>.
            </p></li><li class="listitem"><p>
              <a class="link" href="optimization.html#optimizer-hints-table-level" title="Table-Level Optimizer Hints"><code class="literal">BKA(@qb2)</code></a> applies to
              query block <code class="literal">qb2</code>.
            </p></li><li class="listitem"><p>
              <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_MRR(@qb3 t1 idx1,
              id2)</code></a> applies to indexes <code class="literal">idx1</code>
              and <code class="literal">idx2</code> in table <code class="literal">t1</code>
              in query block <code class="literal">qb3</code>.
</p></li></ul>
</div>
<p>
          Query block names are identifiers and follow the usual rules
          about what names are valid and how to quote them (see
          <a class="xref" href="language-structure.html#identifiers" title="9.2 Schema Object Names">Section 9.2, “Schema Object Names”</a>). For example, a query block
          name that contains spaces must be quoted, which can be done
          using backticks:
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ BKA(@`my hint name`) */ ...
  FROM (SELECT /*+ QB_NAME(`my hint name`) */ ...) ...</pre><p>
          If the <a class="link" href="server-administration.html#sqlmode_ansi_quotes"><code class="literal">ANSI_QUOTES</code></a> SQL mode
          is enabled, it is also possible to quote query block names
          within double quotation marks:
        </p><pre data-lang="sql" class="programlisting">SELECT /*+ BKA(@"my hint name") */ ...
FROM (SELECT /*+ QB_NAME("my hint name") */ ...) ...</pre>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="index-hints"></a>8.9.4 Index Hints</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444347139472"></a><a class="indexterm" name="idm46444347138400"></a><a class="indexterm" name="idm46444347136912"></a><a class="indexterm" name="idm46444347135840"></a><a class="indexterm" name="idm46444347134768"></a><a class="indexterm" name="idm46444347133696"></a><a class="indexterm" name="idm46444347132624"></a><a class="indexterm" name="idm46444347131552"></a><p>
        Index hints give the optimizer information about how to choose
        indexes during query processing. Index hints, described here,
        differ from optimizer hints, described in
        <a class="xref" href="optimization.html#optimizer-hints" title="8.9.3 Optimizer Hints">Section 8.9.3, “Optimizer Hints”</a>. Index and optimizer hints may
        be used separately or together.
      </p><p>
        Index hints apply only to <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>
        and <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a> statements.
      </p><p>
        Index hints are specified following a table name. (For the
        general syntax for specifying tables in a
        <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statement, see
        <a class="xref" href="sql-statements.html#join" title="13.2.10.2 JOIN Clause">Section 13.2.10.2, “JOIN Clause”</a>.) The syntax for referring to an
        individual table, including index hints, looks like this:
      </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>tbl_name</code></em> [[AS] <em class="replaceable"><code>alias</code></em>] [<em class="replaceable"><code>index_hint_list</code></em>]

<em class="replaceable"><code>index_hint_list</code></em>:
    <em class="replaceable"><code>index_hint</code></em> [<em class="replaceable"><code>index_hint</code></em>] ...

<em class="replaceable"><code>index_hint</code></em>:
    USE {INDEX|KEY}
      [FOR {JOIN|ORDER BY|GROUP BY}] ([<em class="replaceable"><code>index_list</code></em>])
  | {IGNORE|FORCE} {INDEX|KEY}
      [FOR {JOIN|ORDER BY|GROUP BY}] (<em class="replaceable"><code>index_list</code></em>)

<em class="replaceable"><code>index_list</code></em>:
    <em class="replaceable"><code>index_name</code></em> [, <em class="replaceable"><code>index_name</code></em>] ...
</pre><p>
        The <code class="literal">USE INDEX
        (<em class="replaceable"><code>index_list</code></em>)</code> hint tells
        MySQL to use only one of the named indexes to find rows in the
        table. The alternative syntax <code class="literal">IGNORE INDEX
        (<em class="replaceable"><code>index_list</code></em>)</code> tells MySQL to
        not use some particular index or indexes. These hints are useful
        if <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> shows that MySQL is
        using the wrong index from the list of possible indexes.
      </p><p>
        The <code class="literal">FORCE INDEX</code> hint acts like <code class="literal">USE
        INDEX (<em class="replaceable"><code>index_list</code></em>)</code>, with
        the addition that a table scan is assumed to be
        <span class="emphasis"><em>very</em></span> expensive. In other words, a table
        scan is used only if there is no way to use one of the named
        indexes to find rows in the table.
</p>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
          As of MySQL 8.0.20, the server supports the index-level
          optimizer hints <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">JOIN_INDEX</code></a>,
          <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">GROUP_INDEX</code></a>,
          <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">ORDER_INDEX</code></a>, and
          <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">INDEX</code></a>, which are equivalent
          to and intended to supersede <code class="literal">FORCE INDEX</code>
          index hints, as well as the
          <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_JOIN_INDEX</code></a>,
          <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_GROUP_INDEX</code></a>,
          <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_ORDER_INDEX</code></a>, and
          <a class="link" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints"><code class="literal">NO_INDEX</code></a> optimizer hints,
          which are equivalent to and intended to supersede
          <code class="literal">IGNORE INDEX</code> index hints. Thus, you should
          expect <code class="literal">USE INDEX</code>, <code class="literal">FORCE
          INDEX</code>, and <code class="literal">IGNORE INDEX</code> to be
          deprecated in a future release of MySQL, and at some time
          thereafter to be removed altogether. For more information, see
          <a class="xref" href="optimization.html#optimizer-hints-index-level" title="Index-Level Optimizer Hints">Index-Level Optimizer Hints</a>.
</p>
</div>
<p>
        Each hint requires index names, not column names. To refer to a
        primary key, use the name <code class="literal">PRIMARY</code>. To see the
        index names for a table, use the <a class="link" href="sql-statements.html#show-index" title="13.7.7.22 SHOW INDEX Statement"><code class="literal">SHOW
        INDEX</code></a> statement or the
        <a class="link" href="information-schema.html#statistics-table" title="25.32 The INFORMATION_SCHEMA STATISTICS Table"><code class="literal">INFORMATION_SCHEMA.STATISTICS</code></a>
        table.
      </p><p>
        An <em class="replaceable"><code>index_name</code></em> value need not be a
        full index name. It can be an unambiguous prefix of an index
        name. If a prefix is ambiguous, an error occurs.
      </p><p>
        Examples:
      </p><pre data-lang="sql" class="programlisting">SELECT * FROM table1 USE INDEX (col1_index,col2_index)
  WHERE col1=1 AND col2=2 AND col3=3;

SELECT * FROM table1 IGNORE INDEX (col3_index)
  WHERE col1=1 AND col2=2 AND col3=3;</pre><p>
        The syntax for index hints has the following characteristics:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            It is syntactically valid to omit
            <em class="replaceable"><code>index_list</code></em> for <code class="literal">USE
            INDEX</code>, which means <span class="quote">“<span class="quote">use no indexes.</span>”</span>
            Omitting <em class="replaceable"><code>index_list</code></em> for
            <code class="literal">FORCE INDEX</code> or <code class="literal">IGNORE
            INDEX</code> is a syntax error.
          </p></li><li class="listitem"><p>
            You can specify the scope of an index hint by adding a
            <code class="literal">FOR</code> clause to the hint. This provides
            more fine-grained control over optimizer selection of an
            execution plan for various phases of query processing. To
            affect only the indexes used when MySQL decides how to find
            rows in the table and how to process joins, use <code class="literal">FOR
            JOIN</code>. To influence index usage for sorting or
            grouping rows, use <code class="literal">FOR ORDER BY</code> or
            <code class="literal">FOR GROUP BY</code>.
          </p></li><li class="listitem"><p>
            You can specify multiple index hints:
          </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 USE INDEX (i1) IGNORE INDEX FOR ORDER BY (i2) ORDER BY a;</pre><p>
            It is not an error to name the same index in several hints
            (even within the same hint):
          </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 USE INDEX (i1) USE INDEX (i1,i1);</pre><p>
            However, it is an error to mix <code class="literal">USE INDEX</code>
            and <code class="literal">FORCE INDEX</code> for the same table:
</p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 USE INDEX FOR JOIN (i1) FORCE INDEX FOR JOIN (i2);</pre></li></ul>
</div>
<p>
        If an index hint includes no <code class="literal">FOR</code> clause, the
        scope of the hint is to apply to all parts of the statement. For
        example, this hint:
      </p><pre data-lang="sql" class="programlisting">IGNORE INDEX (i1)</pre><p>
        is equivalent to this combination of hints:
      </p><pre data-lang="sql" class="programlisting">IGNORE INDEX FOR JOIN (i1)
IGNORE INDEX FOR ORDER BY (i1)
IGNORE INDEX FOR GROUP BY (i1)</pre><p>
        In MySQL 5.0, hint scope with no <code class="literal">FOR</code> clause
        was to apply only to row retrieval. To cause the server to use
        this older behavior when no <code class="literal">FOR</code> clause is
        present, enable the <a class="link" href="server-administration.html#sysvar_old"><code class="literal">old</code></a> system
        variable at server startup. Take care about enabling this
        variable in a replication setup. With statement-based binary
        logging, having different modes for the master and slaves might
        lead to replication errors.
      </p><p>
        When index hints are processed, they are collected in a single
        list by type (<code class="literal">USE</code>, <code class="literal">FORCE</code>,
        <code class="literal">IGNORE</code>) and by scope (<code class="literal">FOR
        JOIN</code>, <code class="literal">FOR ORDER BY</code>, <code class="literal">FOR
        GROUP BY</code>). For example:
      </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1
  USE INDEX () IGNORE INDEX (i2) USE INDEX (i1) USE INDEX (i2);</pre><p>
        is equivalent to:
      </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1
   USE INDEX (i1,i2) IGNORE INDEX (i2);</pre><p>
        The index hints then are applied for each scope in the following
        order:
</p>
<div class="orderedlist">
<ol class="orderedlist" type="1"><li class="listitem"><p>
            <code class="literal">{USE|FORCE} INDEX</code> is applied if present.
            (If not, the optimizer-determined set of indexes is used.)
          </p></li><li class="listitem"><p>
            <code class="literal">IGNORE INDEX</code> is applied over the result
            of the previous step. For example, the following two queries
            are equivalent:
          </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 USE INDEX (i1) IGNORE INDEX (i2) USE INDEX (i2);

SELECT * FROM t1 USE INDEX (i1);</pre></li></ol>
</div>
<p>
        For <code class="literal">FULLTEXT</code> searches, index hints work as
        follows:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            For natural language mode searches, index hints are silently
            ignored. For example, <code class="literal">IGNORE INDEX(i1)</code> is
            ignored with no warning and the index is still used.
          </p></li><li class="listitem"><p>
            For boolean mode searches, index hints with <code class="literal">FOR
            ORDER BY</code> or <code class="literal">FOR GROUP BY</code> are
            silently ignored. Index hints with <code class="literal">FOR
            JOIN</code> or no <code class="literal">FOR</code> modifier are
            honored. In contrast to how hints apply for
            non-<code class="literal">FULLTEXT</code> searches, the hint is used
            for all phases of query execution (finding rows and
            retrieval, grouping, and ordering). This is true even if the
            hint is given for a non-<code class="literal">FULLTEXT</code> index.
          </p><p>
            For example, the following two queries are equivalent:
          </p><pre data-lang="sql" class="programlisting">SELECT * FROM t
  USE INDEX (index1)
  IGNORE INDEX (index1) FOR ORDER BY
  IGNORE INDEX (index1) FOR GROUP BY
  WHERE ... IN BOOLEAN MODE ... ;

SELECT * FROM t
  USE INDEX (index1)
WHERE ... IN BOOLEAN MODE ... ;</pre></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="cost-model"></a>8.9.5 The Optimizer Cost Model</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444347041776"></a><a class="indexterm" name="idm46444347040320"></a><p>
        To generate execution plans, the optimizer uses a cost model
        that is based on estimates of the cost of various operations
        that occur during query execution. The optimizer has a set of
        compiled-in default <span class="quote">“<span class="quote">cost constants</span>”</span> available to
        it to make decisions regarding execution plans.
      </p><p>
        The optimizer also has a database of cost estimates to use
        during execution plan construction. These estimates are stored
        in the <code class="literal">server_cost</code> and
        <code class="literal">engine_cost</code> tables in the
        <code class="literal">mysql</code> system database and are configurable at
        any time. The intent of these tables is to make it possible to
        easily adjust the cost estimates that the optimizer uses when it
        attempts to arrive at query execution plans.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a class="xref" href="optimization.html#cost-model-operation" title="Cost Model General Operation">Cost Model General Operation</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#cost-model-database" title="The Cost Model Database">The Cost Model Database</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#cost-model-database-modifications" title="Making Changes to the Cost Model Database">Making Changes to the Cost Model Database</a></p></li></ul>
</div>

<div class="simplesect">

<div class="titlepage">
<div>

<div class="simple">
<h4 class="title"><a name="cost-model-operation"></a>Cost Model General Operation</h4>

</div>

</div>

</div>
<p>
          The configurable optimizer cost model works like this:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              The server reads the cost model tables into memory at
              startup and uses the in-memory values at runtime. Any
              non-<code class="literal">NULL</code> cost estimate specified in the
              tables takes precedence over the corresponding compiled-in
              default cost constant. Any <code class="literal">NULL</code>
              estimate indicates to the optimizer to use the compiled-in
              default.
            </p></li><li class="listitem"><p>
              At runtime, the server may reread the cost tables. This
              occurs when a storage engine is dynamically loaded or when
              a <a class="link" href="sql-statements.html#flush-optimizer-costs"><code class="literal">FLUSH OPTIMIZER_COSTS</code></a>
              statement is executed.
            </p></li><li class="listitem"><p>
              Cost tables enable server administrators to easily adjust
              cost estimates by changing entries in the tables. It is
              also easy to revert to a default by setting an
              entry's cost to <code class="literal">NULL</code>. The
              optimizer uses the in-memory cost values, so changes to
              the tables should be followed by
              <a class="link" href="sql-statements.html#flush-optimizer-costs"><code class="literal">FLUSH OPTIMIZER_COSTS</code></a> to
              take effect.
            </p></li><li class="listitem"><p>
              The in-memory cost estimates that are current when a
              client session begins apply throughout that session until
              it ends. In particular, if the server rereads the cost
              tables, any changed estimates apply only to subsequently
              started sessions. Existing sessions are unaffected.
            </p></li><li class="listitem"><p>
              Cost tables are specific to a given server instance. The
              server does not replicate cost table changes to
              replication slaves.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="cost-model-database"></a>The Cost Model Database</h4>

</div>

</div>

</div>
<p>
          The optimizer cost model database consists of two tables in
          the <code class="literal">mysql</code> system database that contain cost
          estimate information for operations that occur during query
          execution:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <a class="indexterm" name="idm46444347015760"></a>

              <a class="indexterm" name="idm46444347014272"></a>

              <code class="literal">server_cost</code>: Optimizer cost estimates
              for general server operations
            </p></li><li class="listitem"><p>
              <a class="indexterm" name="idm46444347011232"></a>

              <a class="indexterm" name="idm46444347009744"></a>

              <code class="literal">engine_cost</code>: Optimizer cost estimates
              for operations specific to particular storage engines
</p></li></ul>
</div>
<p>
          The <code class="literal">server_cost</code> table contains these
          columns:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <code class="literal">cost_name</code>
            </p><p>
              The name of a cost estimate used in the cost model. The
              name is not case-sensitive. If the server does not
              recognize the cost name when it reads this table, it
              writes a warning to the error log.
            </p></li><li class="listitem"><p>
              <code class="literal">cost_value</code>
            </p><p>
              The cost estimate value. If the value is
              non-<code class="literal">NULL</code>, the server uses it as the
              cost. Otherwise, it uses the default estimate (the
              compiled-in value). DBAs can change a cost estimate by
              updating this column. If the server finds that the cost
              value is invalid (nonpositive) when it reads this table,
              it writes a warning to the error log.
            </p><p>
              To override a default cost estimate (for an entry that
              specifies <code class="literal">NULL</code>), set the cost to a
              non-<code class="literal">NULL</code> value. To revert to the
              default, set the value to <code class="literal">NULL</code>. Then
              execute <a class="link" href="sql-statements.html#flush-optimizer-costs"><code class="literal">FLUSH
              OPTIMIZER_COSTS</code></a> to tell the server to reread the
              cost tables.
            </p></li><li class="listitem"><p>
              <code class="literal">last_update</code>
            </p><p>
              The time of the last row update.
            </p></li><li class="listitem"><p>
              <code class="literal">comment</code>
            </p><p>
              A descriptive comment associated with the cost estimate.
              DBAs can use this column to provide information about why
              a cost estimate row stores a particular value.
            </p></li><li class="listitem"><p>
              <code class="literal">default_value</code>
            </p><p>
              The default (compiled-in) value for the cost estimate.
              This column is a read-only generated column that retains
              its value even if the associated cost estimate is changed.
              For rows added to the table at runtime, the value of this
              column is <code class="literal">NULL</code>.
</p></li></ul>
</div>
<p>
          The primary key for the <code class="literal">server_cost</code> table
          is the <code class="literal">cost_name</code> column, so it is not
          possible to create multiple entries for any cost estimate.
        </p><p>
          The server recognizes these <code class="literal">cost_name</code>
          values for the <code class="literal">server_cost</code> table:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <code class="literal">disk_temptable_create_cost</code>,
              <code class="literal">disk_temptable_row_cost</code>
            </p><p>
              The cost estimates for internally created temporary tables
              stored in a disk-based storage engine (either
              <code class="literal">InnoDB</code> or <code class="literal">MyISAM</code>).
              Increasing these values increases the cost estimate of
              using internal temporary tables and makes the optimizer
              prefer query plans with less use of them. For information
              about such tables, see
              <a class="xref" href="optimization.html#internal-temporary-tables" title="8.4.4 Internal Temporary Table Use in MySQL">Section 8.4.4, “Internal Temporary Table Use in MySQL”</a>.
            </p><p>
              The larger default values for these disk parameters
              compared to the default values for the corresponding
              memory parameters
              (<code class="literal">memory_temptable_create_cost</code>,
              <code class="literal">memory_temptable_row_cost</code>) reflects the
              greater cost of processing disk-based tables.
            </p></li><li class="listitem"><p>
              <code class="literal">key_compare_cost</code>
            </p><a class="indexterm" name="idm46444346976752"></a><a class="indexterm" name="idm46444346975680"></a><p>
              The cost of comparing record keys. Increasing this value
              causes a query plan that compares many keys to become more
              expensive. For example, a query plan that performs a
              <code class="literal">filesort</code> becomes relatively more
              expensive compared to a query plan that avoids sorting by
              using an index.
            </p></li><li class="listitem"><p>
              <code class="literal">memory_temptable_create_cost</code>,
              <code class="literal">memory_temptable_row_cost</code>
            </p><p>
              The cost estimates for internally created temporary tables
              stored in the <code class="literal">MEMORY</code> storage engine.
              Increasing these values increases the cost estimate of
              using internal temporary tables and makes the optimizer
              prefer query plans with less use of them. For information
              about such tables, see
              <a class="xref" href="optimization.html#internal-temporary-tables" title="8.4.4 Internal Temporary Table Use in MySQL">Section 8.4.4, “Internal Temporary Table Use in MySQL”</a>.
            </p><p>
              The smaller default values for these memory parameters
              compared to the default values for the corresponding disk
              parameters (<code class="literal">disk_temptable_create_cost</code>,
              <code class="literal">disk_temptable_row_cost</code>) reflects the
              lesser cost of processing memory-based tables.
            </p></li><li class="listitem"><p>
              <code class="literal">row_evaluate_cost</code>
            </p><p>
              The cost of evaluating record conditions. Increasing this
              value causes a query plan that examines many rows to
              become more expensive compared to a query plan that
              examines fewer rows. For example, a table scan becomes
              relatively more expensive compared to a range scan that
              reads fewer rows.
</p></li></ul>
</div>
<p>
          The <code class="literal">engine_cost</code> table contains these
          columns:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <code class="literal">engine_name</code>
            </p><p>
              The name of the storage engine to which this cost estimate
              applies. The name is not case-sensitive. If the value is
              <code class="literal">default</code>, it applies to all storage
              engines that have no named entry of their own. If the
              server does not recognize the engine name when it reads
              this table, it writes a warning to the error log.
            </p></li><li class="listitem"><p>
              <code class="literal">device_type</code>
            </p><p>
              The device type to which this cost estimate applies. The
              column is intended for specifying different cost estimates
              for different storage device types, such as hard disk
              drives versus solid state drives. Currently, this
              information is not used and 0 is the only permitted value.
            </p></li><li class="listitem"><p>
              <code class="literal">cost_name</code>
            </p><p>
              Same as in the <code class="literal">server_cost</code> table.
            </p></li><li class="listitem"><p>
              <code class="literal">cost_value</code>
            </p><p>
              Same as in the <code class="literal">server_cost</code> table.
            </p></li><li class="listitem"><p>
              <code class="literal">last_update</code>
            </p><p>
              Same as in the <code class="literal">server_cost</code> table.
            </p></li><li class="listitem"><p>
              <code class="literal">comment</code>
            </p><p>
              Same as in the <code class="literal">server_cost</code> table.
            </p></li><li class="listitem"><p>
              <code class="literal">default_value</code>
            </p><p>
              The default (compiled-in) value for the cost estimate.
              This column is a read-only generated column that retains
              its value even if the associated cost estimate is changed.
              For rows added to the table at runtime, the value of this
              column is <code class="literal">NULL</code>, with the exception that
              if the row has the same <code class="literal">cost_name</code> value
              as one of the original rows, the
              <code class="literal">default_value</code> column will have the same
              value as that row.
</p></li></ul>
</div>
<p>
          The primary key for the <code class="literal">engine_cost</code> table
          is a tuple comprising the (<code class="literal">cost_name</code>,
          <code class="literal">engine_name</code>,
          <code class="literal">device_type</code>) columns, so it is not possible
          to create multiple entries for any combination of values in
          those columns.
        </p><p>
          The server recognizes these <code class="literal">cost_name</code>
          values for the <code class="literal">engine_cost</code> table:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <code class="literal">io_block_read_cost</code>
            </p><p>
              The cost of reading an index or data block from disk.
              Increasing this value causes a query plan that reads many
              disk blocks to become more expensive compared to a query
              plan that reads fewer disk blocks. For example, a table
              scan becomes relatively more expensive compared to a range
              scan that reads fewer blocks.
            </p></li><li class="listitem"><p>
              <code class="literal">memory_block_read_cost</code>
            </p><p>
              Similar to <code class="literal">io_block_read_cost</code>, but
              represents the cost of reading an index or data block from
              an in-memory database buffer.
</p></li></ul>
</div>
<p>
          If the <code class="literal">io_block_read_cost</code> and
          <code class="literal">memory_block_read_cost</code> values differ, the
          execution plan may change between two runs of the same query.
          Suppose that the cost for memory access is less than the cost
          for disk access. In that case, at server startup before data
          has been read into the buffer pool, you may get a different
          plan than after the query has been run because then the data
          will be in memory.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="cost-model-database-modifications"></a>Making Changes to the Cost Model Database</h4>

</div>

</div>

</div>
<p>
          For DBAs who wish to change the cost model parameters from
          their defaults, try doubling or halving the value and
          measuring the effect.
        </p><p>
          Changes to the <code class="literal">io_block_read_cost</code> and
          <code class="literal">memory_block_read_cost</code> parameters are most
          likely to yield worthwhile results. These parameter values
          enable cost models for data access methods to take into
          account the costs of reading information from different
          sources; that is, the cost of reading information from disk
          versus reading information already in a memory buffer. For
          example, all other things being equal, setting
          <code class="literal">io_block_read_cost</code> to a value larger than
          <code class="literal">memory_block_read_cost</code> causes the optimizer
          to prefer query plans that read information already held in
          memory to plans that must read from disk.
        </p><p>
          This example shows how to change the default value for
          <code class="literal">io_block_read_cost</code>:
        </p><pre data-lang="sql" class="programlisting">UPDATE mysql.engine_cost
  SET cost_value = 2.0
  WHERE cost_name = 'io_block_read_cost';
FLUSH OPTIMIZER_COSTS;</pre><p>
          This example shows how to change the value of
          <code class="literal">io_block_read_cost</code> only for the
          <code class="literal">InnoDB</code> storage engine:
        </p><pre data-lang="sql" class="programlisting">INSERT INTO mysql.engine_cost
  VALUES ('InnoDB', 0, 'io_block_read_cost', 3.0,
  CURRENT_TIMESTAMP, 'Using a slower disk for InnoDB');
FLUSH OPTIMIZER_COSTS;</pre>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="optimizer-statistics"></a>8.9.6 Optimizer Statistics</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444346916736"></a><a class="indexterm" name="idm46444346915248"></a><p>
        The <code class="literal">column_statistics</code> data dictionary table
        stores histogram statistics about column values, for use by the
        optimizer in constructing query execution plans. To perform
        histogram management, use the <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE
        TABLE</code></a> statement; see <a class="xref" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement">Section 13.7.3.1, “ANALYZE TABLE Statement”</a>.
      </p><p>
        The <code class="literal">column_statistics</code> table has these
        characteristics:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            The table contains statistics for columns of all data types
            except geometry types (spatial data) and
            <a class="link" href="data-types.html#json" title="11.5 The JSON Data Type"><code class="literal">JSON</code></a>.
          </p></li><li class="listitem"><p>
            The table is persistent so that column statistics need not
            be created each time the server starts.
          </p></li><li class="listitem"><p>
            The server performs updates to the table; users do not.
</p></li></ul>
</div>
<p>
        The <code class="literal">column_statistics</code> table is not directly
        accessible by users because it is part of the data dictionary.
        Histogram information is available using
        <a class="link" href="information-schema.html#column-statistics-table" title="25.10 The INFORMATION_SCHEMA COLUMN_STATISTICS Table"><code class="literal">INFORMATION_SCHEMA.COLUMN_STATISTICS</code></a>,
        which is implemented as a view on the data dictionary table.
        <a class="link" href="information-schema.html#column-statistics-table" title="25.10 The INFORMATION_SCHEMA COLUMN_STATISTICS Table"><code class="literal">COLUMN_STATISTICS</code></a> has these
        columns:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            <code class="literal">SCHEMA_NAME</code>,
            <code class="literal">TABLE_NAME</code>,
            <code class="literal">COLUMN_NAME</code>: The names of the schema,
            table, and column for which the statistics apply.
          </p></li><li class="listitem"><p>
            <code class="literal">HISTOGRAM</code>: A
            <a class="link" href="data-types.html#json" title="11.5 The JSON Data Type"><code class="literal">JSON</code></a> value describing the
            column statistics, stored as a histogram.
</p></li></ul>
</div>
<p>
        Column histograms contain buckets for parts of the range of
        values stored in the column. Histograms are
        <a class="link" href="data-types.html#json" title="11.5 The JSON Data Type"><code class="literal">JSON</code></a> objects to permit
        flexibility in the representation of column statistics. Here is
        a sample histogram object:
      </p><pre data-lang="json" class="programlisting">{
  "buckets": [
    [
      1,
      0.3333333333333333
    ],
    [
      2,
      0.6666666666666666
    ],
    [
      3,
      1
    ]
  ],
  "null-values": 0,
  "last-updated": "2017-03-24 13:32:40.000000",
  "sampling-rate": 1,
  "histogram-type": "singleton",
  "number-of-buckets-specified": 128,
  "data-type": "int",
  "collation-id": 8
}</pre><p>
        Histogram objects have these keys:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            <code class="literal">buckets</code>: The histogram buckets. Bucket
            structure depends on the histogram type.
          </p><p>
            For <code class="literal">singleton</code> histograms, buckets contain
            two values:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                Value 1: The value for the bucket. The type depends on
                the column data type.
              </p></li><li class="listitem"><p>
                Value 2: A double representing the cumulative frequency
                for the value. For example, .25 and .75 indicate that
                25% and 75% of the values in the column are less than or
                equal to the bucket value.
</p></li></ul>
</div>
<p>
            For <code class="literal">equi-height</code> histograms, buckets
            contain four values:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                Values 1, 2: The lower and upper inclusive values for
                the bucket. The type depends on the column data type.
              </p></li><li class="listitem"><p>
                Value 3: A double representing the cumulative frequency
                for the value. For example, .25 and .75 indicate that
                25% and 75% of the values in the column are less than or
                equal to the bucket upper value.
              </p></li><li class="listitem"><p>
                Value 4: The number of distinct values in the range from
                the bucket lower value to its upper value.
</p></li></ul>
</div>
</li><li class="listitem"><p>
            <code class="literal">null-values</code>: A number between 0.0 and 1.0
            indicating the fraction of column values that are SQL
            <code class="literal">NULL</code> values. If 0, the column contains no
            <code class="literal">NULL</code> values.
          </p></li><li class="listitem"><p>
            <code class="literal">last-updated</code>: When the histogram was
            generated, as a UTC value in <em class="replaceable"><code>YYYY-MM-DD
            hh:mm:ss.uuuuuu</code></em> format.
          </p></li><li class="listitem"><p>
            <code class="literal">sampling-rate</code>: A number between 0.0 and
            1.0 indicating the fraction of data that was sampled to
            create the histogram. A value of 1 means that all of the
            data was read (no sampling).
          </p></li><li class="listitem"><p>
            <code class="literal">histogram-type</code>: The histogram type:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">singleton</code>: One bucket represents one
                single value in the column. This histogram type is
                created when the number of distinct values in the column
                is less than or equal to the number of buckets specified
                in the <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE TABLE</code></a>
                statement that generated the histogram.
              </p></li><li class="listitem"><p>
                <code class="literal">equi-height</code>: One bucket represents a
                range of values. This histogram type is created when the
                number of distinct values in the column is greater than
                the number of buckets specified in the
                <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE TABLE</code></a> statement
                that generated the histogram.
</p></li></ul>
</div>
</li><li class="listitem"><p>
            <code class="literal">number-of-buckets-specified</code>: The number
            of buckets specified in the <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE
            TABLE</code></a> statement that generated the histogram.
          </p></li><li class="listitem"><p>
            <code class="literal">data-type</code>: The type of data this
            histogram contains. This is needed when reading and parsing
            histograms from persistent storage into memory. The value is
            one of <code class="literal">int</code>, <code class="literal">uint</code>
            (unsigned integer), <code class="literal">double</code>,
            <code class="literal">decimal</code>, <code class="literal">datetime</code>, or
            <code class="literal">string</code> (includes character and binary
            strings).
          </p></li><li class="listitem"><p>
            <code class="literal">collation-id</code>: The collation ID for the
            histogram data. It is mostly meaningful when the
            <code class="literal">data-type</code> value is
            <code class="literal">string</code>. Values correspond to
            <code class="literal">ID</code> column values in the
            <a class="link" href="information-schema.html#collations-table" title="25.6 The INFORMATION_SCHEMA COLLATIONS Table"><code class="literal">INFORMATION_SCHEMA.COLLATIONS</code></a>
            table.
</p></li></ul>
</div>
<p>
        To extract particular values from the histogram objects, you can
        use <a class="link" href="data-types.html#json" title="11.5 The JSON Data Type"><code class="literal">JSON</code></a> operations. For example:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT</code></strong>
         <strong class="userinput"><code>TABLE_NAME, COLUMN_NAME,</code></strong>
         <strong class="userinput"><code>HISTOGRAM-&gt;&gt;'$."data-type"' AS 'data-type',</code></strong>
         <strong class="userinput"><code>JSON_LENGTH(HISTOGRAM-&gt;&gt;'$."buckets"') AS 'bucket-count'</code></strong>
       <strong class="userinput"><code>FROM INFORMATION_SCHEMA.COLUMN_STATISTICS;</code></strong>
+-----------------+-------------+-----------+--------------+
| TABLE_NAME      | COLUMN_NAME | data-type | bucket-count |
+-----------------+-------------+-----------+--------------+
| country         | Population  | int       |          226 |
| city            | Population  | int       |         1024 |
| countrylanguage | Language    | string    |          457 |
+-----------------+-------------+-----------+--------------+
</pre><p>
        The optimizer uses histogram statistics, if applicable, for
        columns of any data type for which statistics are collected. The
        optimizer applies histogram statistics to determine row
        estimates based on the selectivity (filtering effect) of column
        value comparisons against constant values. Predicates of these
        forms qualify for histogram use:
      </p><pre data-lang="sql" class="programlisting"><em class="replaceable"><code>col_name</code></em> = <em class="replaceable"><code>constant</code></em>
<em class="replaceable"><code>col_name</code></em> &lt;&gt; <em class="replaceable"><code>constant</code></em>
<em class="replaceable"><code>col_name</code></em> != <em class="replaceable"><code>constant</code></em>
<em class="replaceable"><code>col_name</code></em> &gt; <em class="replaceable"><code>constant</code></em>
<em class="replaceable"><code>col_name</code></em> &lt; <em class="replaceable"><code>constant</code></em>
<em class="replaceable"><code>col_name</code></em> &gt;= <em class="replaceable"><code>constant</code></em>
<em class="replaceable"><code>col_name</code></em> &lt;= <em class="replaceable"><code>constant</code></em>
<em class="replaceable"><code>col_name</code></em> IS NULL
<em class="replaceable"><code>col_name</code></em> IS NOT NULL
<em class="replaceable"><code>col_name</code></em> BETWEEN <em class="replaceable"><code>constant</code></em> AND <em class="replaceable"><code>constant</code></em>
<em class="replaceable"><code>col_name</code></em> NOT BETWEEN <em class="replaceable"><code>constant</code></em> AND <em class="replaceable"><code>constant</code></em>
<em class="replaceable"><code>col_name</code></em> IN (<em class="replaceable"><code>constant</code></em>[, <em class="replaceable"><code>constant</code></em>] ...)
<em class="replaceable"><code>col_name</code></em> NOT IN (<em class="replaceable"><code>constant</code></em>[, <em class="replaceable"><code>constant</code></em>] ...)
</pre><p>
        For example, these statements contain predicates that qualify
        for histogram use:
      </p><pre data-lang="sql" class="programlisting">SELECT * FROM orders WHERE amount BETWEEN 100.0 AND 300.0;
SELECT * FROM tbl WHERE col1 = 15 AND col2 &gt; 100;</pre><p>
        The requirement for comparison against a constant value includes
        functions that are constant, such as
        <a class="link" href="functions.html#function_abs"><code class="literal">ABS()</code></a> and
        <a class="link" href="functions.html#function_floor"><code class="literal">FLOOR()</code></a>:
      </p><pre data-lang="sql" class="programlisting">SELECT * FROM tbl WHERE col1 &lt; ABS(-34);</pre><p>
        Histogram statistics are useful primarily for nonindexed
        columns. Adding an index to a column for which histogram
        statistics are applicable might also help the optimizer make row
        estimates. The tradeoffs are:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            An index must be updated when table data is modified.
          </p></li><li class="listitem"><p>
            A histogram is created or updated only on demand, so it adds
            no overhead when table data is modified. On the other hand,
            the statistics become progressively more out of date when
            table modifications occur, until the next time they are
            updated.
</p></li></ul>
</div>
<p>
        The optimizer prefers range optimizer row estimates to those
        obtained from histogram statistics. If the optimizer determines
        that the range optimizer applies, it does not use histogram
        statistics.
      </p><p>
        For columns that are indexed, row estimates can be obtained for
        equality comparisons using index dives (see
        <a class="xref" href="optimization.html#range-optimization" title="8.2.1.2 Range Optimization">Section 8.2.1.2, “Range Optimization”</a>). In this case, histogram
        statistics are not necessarily useful because index dives can
        yield better estimates.
      </p><p>
        In some cases, use of histogram statistics may not improve query
        execution (for example, if the statistics are out of date). To
        check whether this is the case, use <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE
        TABLE</code></a> to regenerate the histogram statistics, then run
        the query again.
      </p><p>
        Alternatively, to disable histogram statistics, use
        <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE TABLE</code></a> to drop them. A
        different method of disabling histogram statistics is to turn
        off the <code class="literal">condition_fanout_filter</code> flag of the
        <a class="link" href="server-administration.html#sysvar_optimizer_switch"><code class="literal">optimizer_switch</code></a> system
        variable (although this may disable other optimizations as
        well):
      </p><pre data-lang="sql" class="programlisting">SET optimizer_switch='condition_fanout_filter=off';</pre><p>
        If histogram statistics are used, the resulting effect is
        visible using <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a>. Consider
        the following query, where no index is available for column
        <code class="literal">col1</code>:
      </p><pre data-lang="sql" class="programlisting">SELECT * FROM t1 WHERE col1 &lt; 24;</pre><p>
        If histogram statistics indicate that 57% of the rows in
        <code class="literal">t1</code> satisfy the <code class="literal">col1 &lt;
        24</code> predicate, filtering can occur even in the absence
        of an index, and <a class="link" href="sql-statements.html#explain" title="13.8.2 EXPLAIN Statement"><code class="literal">EXPLAIN</code></a> shows
        57.00 in the <code class="literal">filtered</code> column.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a name="buffering-caching"></a>8.10 Buffering and Caching</h2>

</div>

</div>

</div>
<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#innodb-buffer-pool-optimization">8.10.1 InnoDB Buffer Pool Optimization</a></span></dt><dt><span class="section"><a href="optimization.html#myisam-key-cache">8.10.2 The MyISAM Key Cache</a></span></dt><dt><span class="section"><a href="optimization.html#statement-caching">8.10.3 Caching of Prepared Statements and Stored Programs</a></span></dt></dl>
</div>
<p>
      MySQL uses several strategies that cache information in memory
      buffers to increase performance.
</p>
<div class="section">

<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="innodb-buffer-pool-optimization"></a>8.10.1 InnoDB Buffer Pool Optimization</h3>
</div>
</div>
</div>
<a class="indexterm" name="idm46444346804000"></a><a class="indexterm" name="idm46444346802928"></a><a class="indexterm" name="idm46444346801856"></a><p>
        <a class="link" href="innodb-storage-engine.html" title="Chapter 15 The InnoDB Storage Engine"><code class="literal">InnoDB</code></a> maintains a storage area
        called the <a class="link" href="glossary.html#glos_buffer_pool" title="buffer pool">buffer pool</a>
        for caching data and indexes in memory. Knowing how the
        <code class="literal">InnoDB</code> buffer pool works, and taking
        advantage of it to keep frequently accessed data in memory, is
        an important aspect of MySQL tuning.
      </p><p>
        For an explanation of the inner workings of the
        <code class="literal">InnoDB</code> buffer pool, an overview of its LRU
        replacement algorithm, and general configuration information,
        see <a class="xref" href="innodb-storage-engine.html#innodb-buffer-pool" title="15.5.1 Buffer Pool">Section 15.5.1, “Buffer Pool”</a>.
      </p><p>
        For additional <code class="literal">InnoDB</code> buffer pool
        configuration and tuning information, see these sections:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            <a class="xref" href="innodb-storage-engine.html#innodb-performance-read_ahead" title="15.8.3.4 Configuring InnoDB Buffer Pool Prefetching (Read-Ahead)">Section 15.8.3.4, “Configuring InnoDB Buffer Pool Prefetching (Read-Ahead)”</a>
          </p></li><li class="listitem"><p>
            <a class="xref" href="innodb-storage-engine.html#innodb-buffer-pool-flushing" title="15.8.3.5 Configuring Buffer Pool Flushing">Section 15.8.3.5, “Configuring Buffer Pool Flushing”</a>
          </p></li><li class="listitem"><p>
            <a class="xref" href="innodb-storage-engine.html#innodb-performance-midpoint_insertion" title="15.8.3.3 Making the Buffer Pool Scan Resistant">Section 15.8.3.3, “Making the Buffer Pool Scan Resistant”</a>
          </p></li><li class="listitem"><p>
            <a class="xref" href="innodb-storage-engine.html#innodb-multiple-buffer-pools" title="15.8.3.2 Configuring Multiple Buffer Pool Instances">Section 15.8.3.2, “Configuring Multiple Buffer Pool Instances”</a>
          </p></li><li class="listitem"><p>
            <a class="xref" href="innodb-storage-engine.html#innodb-preload-buffer-pool" title="15.8.3.6 Saving and Restoring the Buffer Pool State">Section 15.8.3.6, “Saving and Restoring the Buffer Pool State”</a>
          </p></li><li class="listitem"><p>
            <a class="xref" href="innodb-storage-engine.html#innodb-buffer-pool-resize" title="15.8.3.1 Configuring InnoDB Buffer Pool Size">Section 15.8.3.1, “Configuring InnoDB Buffer Pool Size”</a>
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="myisam-key-cache"></a>8.10.2 The MyISAM Key Cache</h3>

</div>

</div>

</div>
<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#shared-key-cache">8.10.2.1 Shared Key Cache Access</a></span></dt><dt><span class="section"><a href="optimization.html#multiple-key-caches">8.10.2.2 Multiple Key Caches</a></span></dt><dt><span class="section"><a href="optimization.html#midpoint-insertion">8.10.2.3 Midpoint Insertion Strategy</a></span></dt><dt><span class="section"><a href="optimization.html#index-preloading">8.10.2.4 Index Preloading</a></span></dt><dt><span class="section"><a href="optimization.html#key-cache-block-size">8.10.2.5 Key Cache Block Size</a></span></dt><dt><span class="section"><a href="optimization.html#key-cache-restructuring">8.10.2.6 Restructuring a Key Cache</a></span></dt></dl>
</div>
<a class="indexterm" name="idm46444346783552"></a><a class="indexterm" name="idm46444346782480"></a><p>
        To minimize disk I/O, the <code class="literal">MyISAM</code> storage
        engine exploits a strategy that is used by many database
        management systems. It employs a cache mechanism to keep the
        most frequently accessed table blocks in memory:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            For index blocks, a special structure called the
            <span class="firstterm">key cache</span> (or
            <span class="firstterm">key buffer</span>) is
            maintained. The structure contains a number of block buffers
            where the most-used index blocks are placed.
          </p></li><li class="listitem"><p>
            For data blocks, MySQL uses no special cache. Instead it
            relies on the native operating system file system cache.
</p></li></ul>
</div>
<p>
        This section first describes the basic operation of the
        <code class="literal">MyISAM</code> key cache. Then it discusses features
        that improve key cache performance and that enable you to better
        control cache operation:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Multiple sessions can access the cache concurrently.
          </p></li><li class="listitem"><p>
            You can set up multiple key caches and assign table indexes
            to specific caches.
</p></li></ul>
</div>
<p>
        To control the size of the key cache, use the
        <a class="link" href="server-administration.html#sysvar_key_buffer_size"><code class="literal">key_buffer_size</code></a> system
        variable. If this variable is set equal to zero, no key cache is
        used. The key cache also is not used if the
        <a class="link" href="server-administration.html#sysvar_key_buffer_size"><code class="literal">key_buffer_size</code></a> value is too
        small to allocate the minimal number of block buffers (8).
      </p><p>
        When the key cache is not operational, index files are accessed
        using only the native file system buffering provided by the
        operating system. (In other words, table index blocks are
        accessed using the same strategy as that employed for table data
        blocks.)
      </p><p>
        An index block is a contiguous unit of access to the
        <code class="literal">MyISAM</code> index files. Usually the size of an
        index block is equal to the size of nodes of the index B-tree.
        (Indexes are represented on disk using a B-tree data structure.
        Nodes at the bottom of the tree are leaf nodes. Nodes above the
        leaf nodes are nonleaf nodes.)
      </p><p>
        All block buffers in a key cache structure are the same size.
        This size can be equal to, greater than, or less than the size
        of a table index block. Usually one these two values is a
        multiple of the other.
      </p><p>
        When data from any table index block must be accessed, the
        server first checks whether it is available in some block buffer
        of the key cache. If it is, the server accesses data in the key
        cache rather than on disk. That is, it reads from the cache or
        writes into it rather than reading from or writing to disk.
        Otherwise, the server chooses a cache block buffer containing a
        different table index block (or blocks) and replaces the data
        there by a copy of required table index block. As soon as the
        new index block is in the cache, the index data can be accessed.
      </p><p>
        If it happens that a block selected for replacement has been
        modified, the block is considered <span class="quote">“<span class="quote">dirty.</span>”</span> In this
        case, prior to being replaced, its contents are flushed to the
        table index from which it came.
      </p><p>
        Usually the server follows an <span class="firstterm">LRU
        (Least Recently Used)</span> strategy: When choosing a block
        for replacement, it selects the least recently used index block.
        To make this choice easier, the key cache module maintains all
        used blocks in a special list (<span class="firstterm">LRU
        chain</span>) ordered by time of use. When a block is
        accessed, it is the most recently used and is placed at the end
        of the list. When blocks need to be replaced, blocks at the
        beginning of the list are the least recently used and become the
        first candidates for eviction.
      </p><p>
        The <code class="literal">InnoDB</code> storage engine also uses an LRU
        algorithm, to manage its buffer pool. See
        <a class="xref" href="innodb-storage-engine.html#innodb-buffer-pool" title="15.5.1 Buffer Pool">Section 15.5.1, “Buffer Pool”</a>.
</p>
<div class="section">

<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="shared-key-cache"></a>8.10.2.1 Shared Key Cache Access</h4>
</div>
</div>
</div>
<p>
          Threads can access key cache buffers simultaneously, subject
          to the following conditions:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              A buffer that is not being updated can be accessed by
              multiple sessions.
            </p></li><li class="listitem"><p>
              A buffer that is being updated causes sessions that need
              to use it to wait until the update is complete.
            </p></li><li class="listitem"><p>
              Multiple sessions can initiate requests that result in
              cache block replacements, as long as they do not interfere
              with each other (that is, as long as they need different
              index blocks, and thus cause different cache blocks to be
              replaced).
</p></li></ul>
</div>
<p>
          Shared access to the key cache enables the server to improve
          throughput significantly.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="multiple-key-caches"></a>8.10.2.2 Multiple Key Caches</h4>

</div>

</div>

</div>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">
<div class="admon-title">
Note
</div>
<p>
            As of MySQL 8.0, the compound-part
            structured-variable syntax discussed here for referring to
            multiple <code class="literal">MyISAM</code> key caches is deprecated.
</p>
</div>
<p>
          Shared access to the key cache improves performance but does
          not eliminate contention among sessions entirely. They still
          compete for control structures that manage access to the key
          cache buffers. To reduce key cache access contention further,
          MySQL also provides multiple key caches. This feature enables
          you to assign different table indexes to different key caches.
        </p><p>
          Where there are multiple key caches, the server must know
          which cache to use when processing queries for a given
          <code class="literal">MyISAM</code> table. By default, all
          <code class="literal">MyISAM</code> table indexes are cached in the
          default key cache. To assign table indexes to a specific key
          cache, use the <a class="link" href="sql-statements.html#cache-index" title="13.7.8.2 CACHE INDEX Statement"><code class="literal">CACHE INDEX</code></a>
          statement (see <a class="xref" href="sql-statements.html#cache-index" title="13.7.8.2 CACHE INDEX Statement">Section 13.7.8.2, “CACHE INDEX Statement”</a>). For example,
          the following statement assigns indexes from the tables
          <code class="literal">t1</code>, <code class="literal">t2</code>, and
          <code class="literal">t3</code> to the key cache named
          <code class="literal">hot_cache</code>:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>CACHE INDEX t1, t2, t3 IN hot_cache;</code></strong>
+---------+--------------------+----------+----------+
| Table   | Op                 | Msg_type | Msg_text |
+---------+--------------------+----------+----------+
| test.t1 | assign_to_keycache | status   | OK       |
| test.t2 | assign_to_keycache | status   | OK       |
| test.t3 | assign_to_keycache | status   | OK       |
+---------+--------------------+----------+----------+
</pre><p>
          The key cache referred to in a <a class="link" href="sql-statements.html#cache-index" title="13.7.8.2 CACHE INDEX Statement"><code class="literal">CACHE
          INDEX</code></a> statement can be created by setting its size
          with a <a class="link" href="sql-statements.html#set-variable" title="13.7.6.1 SET Syntax for Variable Assignment"><code class="literal">SET
          GLOBAL</code></a> parameter setting statement or by using
          server startup options. For example:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SET GLOBAL keycache1.key_buffer_size=128*1024;</code></strong>
</pre><p>
          To destroy a key cache, set its size to zero:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SET GLOBAL keycache1.key_buffer_size=0;</code></strong>
</pre><p>
          You cannot destroy the default key cache. Any attempt to do
          this is ignored:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SET GLOBAL key_buffer_size = 0;</code></strong>

mysql&gt; <strong class="userinput"><code>SHOW VARIABLES LIKE 'key_buffer_size';</code></strong>
+-----------------+---------+
| Variable_name   | Value   |
+-----------------+---------+
| key_buffer_size | 8384512 |
+-----------------+---------+
</pre><p>
          Key cache variables are structured system variables that have
          a name and components. For
          <code class="literal">keycache1.key_buffer_size</code>,
          <code class="literal">keycache1</code> is the cache variable name and
          <a class="link" href="server-administration.html#sysvar_key_buffer_size"><code class="literal">key_buffer_size</code></a> is the cache
          component. See <a class="xref" href="server-administration.html#structured-system-variables" title="5.1.9.5 Structured System Variables">Section 5.1.9.5, “Structured System Variables”</a>,
          for a description of the syntax used for referring to
          structured key cache system variables.
        </p><p>
          By default, table indexes are assigned to the main (default)
          key cache created at the server startup. When a key cache is
          destroyed, all indexes assigned to it are reassigned to the
          default key cache.
        </p><p>
          For a busy server, you can use a strategy that involves three
          key caches:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              A <span class="quote">“<span class="quote">hot</span>”</span> key cache that takes up 20% of the
              space allocated for all key caches. Use this for tables
              that are heavily used for searches but that are not
              updated.
            </p></li><li class="listitem"><p>
              A <span class="quote">“<span class="quote">cold</span>”</span> key cache that takes up 20% of the
              space allocated for all key caches. Use this cache for
              medium-sized, intensively modified tables, such as
              temporary tables.
            </p></li><li class="listitem"><p>
              A <span class="quote">“<span class="quote">warm</span>”</span> key cache that takes up 60% of the
              key cache space. Employ this as the default key cache, to
              be used by default for all other tables.
</p></li></ul>
</div>
<p>
          One reason the use of three key caches is beneficial is that
          access to one key cache structure does not block access to the
          others. Statements that access tables assigned to one cache do
          not compete with statements that access tables assigned to
          another cache. Performance gains occur for other reasons as
          well:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              The hot cache is used only for retrieval queries, so its
              contents are never modified. Consequently, whenever an
              index block needs to be pulled in from disk, the contents
              of the cache block chosen for replacement need not be
              flushed first.
            </p></li><li class="listitem"><p>
              For an index assigned to the hot cache, if there are no
              queries requiring an index scan, there is a high
              probability that the index blocks corresponding to nonleaf
              nodes of the index B-tree remain in the cache.
            </p></li><li class="listitem"><p>
              An update operation most frequently executed for temporary
              tables is performed much faster when the updated node is
              in the cache and need not be read from disk first. If the
              size of the indexes of the temporary tables are comparable
              with the size of cold key cache, the probability is very
              high that the updated node is in the cache.
</p></li></ul>
</div>
<p>
          The <a class="link" href="sql-statements.html#cache-index" title="13.7.8.2 CACHE INDEX Statement"><code class="literal">CACHE INDEX</code></a> statement sets
          up an association between a table and a key cache, but the
          association is lost each time the server restarts. If you want
          the association to take effect each time the server starts,
          one way to accomplish this is to use an option file: Include
          variable settings that configure your key caches, and an
          <a class="link" href="server-administration.html#sysvar_init_file"><code class="literal">init_file</code></a> system variable
          that names a file containing <a class="link" href="sql-statements.html#cache-index" title="13.7.8.2 CACHE INDEX Statement"><code class="literal">CACHE
          INDEX</code></a> statements to be executed. For example:
        </p><pre data-lang="ini" class="programlisting">key_buffer_size = 4G
hot_cache.key_buffer_size = 2G
cold_cache.key_buffer_size = 2G
init_file=/<em class="replaceable"><code>path</code></em>/<em class="replaceable"><code>to</code></em>/<em class="replaceable"><code>data-directory</code></em>/mysqld_init.sql
</pre><p>
          The statements in <code class="filename">mysqld_init.sql</code> are
          executed each time the server starts. The file should contain
          one SQL statement per line. The following example assigns
          several tables each to <code class="literal">hot_cache</code> and
          <code class="literal">cold_cache</code>:
        </p><pre data-lang="sql" class="programlisting">CACHE INDEX db1.t1, db1.t2, db2.t3 IN hot_cache
CACHE INDEX db1.t4, db2.t5, db2.t6 IN cold_cache</pre>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="midpoint-insertion"></a>8.10.2.3 Midpoint Insertion Strategy</h4>

</div>

</div>

</div>
<p>
          By default, the key cache management system uses a simple LRU
          strategy for choosing key cache blocks to be evicted, but it
          also supports a more sophisticated method called the
          <span class="firstterm">midpoint insertion
          strategy.</span>
        </p><p>
          When using the midpoint insertion strategy, the LRU chain is
          divided into two parts: a hot sublist and a warm sublist. The
          division point between two parts is not fixed, but the key
          cache management system takes care that the warm part is not
          <span class="quote">“<span class="quote">too short,</span>”</span> always containing at least
          <a class="link" href="server-administration.html#sysvar_key_cache_division_limit"><code class="literal">key_cache_division_limit</code></a>
          percent of the key cache blocks.
          <a class="link" href="server-administration.html#sysvar_key_cache_division_limit"><code class="literal">key_cache_division_limit</code></a> is a
          component of structured key cache variables, so its value is a
          parameter that can be set per cache.
        </p><p>
          When an index block is read from a table into the key cache,
          it is placed at the end of the warm sublist. After a certain
          number of hits (accesses of the block), it is promoted to the
          hot sublist. At present, the number of hits required to
          promote a block (3) is the same for all index blocks.
        </p><p>
          A block promoted into the hot sublist is placed at the end of
          the list. The block then circulates within this sublist. If
          the block stays at the beginning of the sublist for a long
          enough time, it is demoted to the warm sublist. This time is
          determined by the value of the
          <a class="link" href="server-administration.html#sysvar_key_cache_age_threshold"><code class="literal">key_cache_age_threshold</code></a>
          component of the key cache.
        </p><p>
          The threshold value prescribes that, for a key cache
          containing <em class="replaceable"><code>N</code></em> blocks, the block at
          the beginning of the hot sublist not accessed within the last
          <code class="literal"><em class="replaceable"><code>N</code></em> *
          key_cache_age_threshold / 100</code> hits is to be moved to
          the beginning of the warm sublist. It then becomes the first
          candidate for eviction, because blocks for replacement always
          are taken from the beginning of the warm sublist.
        </p><p>
          The midpoint insertion strategy enables you to keep
          more-valued blocks always in the cache. If you prefer to use
          the plain LRU strategy, leave the
          <a class="link" href="server-administration.html#sysvar_key_cache_division_limit"><code class="literal">key_cache_division_limit</code></a>
          value set to its default of 100.
        </p><p>
          The midpoint insertion strategy helps to improve performance
          when execution of a query that requires an index scan
          effectively pushes out of the cache all the index blocks
          corresponding to valuable high-level B-tree nodes. To avoid
          this, you must use a midpoint insertion strategy with the
          <a class="link" href="server-administration.html#sysvar_key_cache_division_limit"><code class="literal">key_cache_division_limit</code></a> set
          to much less than 100. Then valuable frequently hit nodes are
          preserved in the hot sublist during an index scan operation as
          well.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="index-preloading"></a>8.10.2.4 Index Preloading</h4>

</div>

</div>

</div>
<p>
          If there are enough blocks in a key cache to hold blocks of an
          entire index, or at least the blocks corresponding to its
          nonleaf nodes, it makes sense to preload the key cache with
          index blocks before starting to use it. Preloading enables you
          to put the table index blocks into a key cache buffer in the
          most efficient way: by reading the index blocks from disk
          sequentially.
        </p><p>
          Without preloading, the blocks are still placed into the key
          cache as needed by queries. Although the blocks will stay in
          the cache, because there are enough buffers for all of them,
          they are fetched from disk in random order, and not
          sequentially.
        </p><p>
          To preload an index into a cache, use the
          <a class="link" href="sql-statements.html#load-index" title="13.7.8.5 LOAD INDEX INTO CACHE Statement"><code class="literal">LOAD INDEX INTO
          CACHE</code></a> statement. For example, the following
          statement preloads nodes (index blocks) of indexes of the
          tables <code class="literal">t1</code> and <code class="literal">t2</code>:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>LOAD INDEX INTO CACHE t1, t2 IGNORE LEAVES;</code></strong>
+---------+--------------+----------+----------+
| Table   | Op           | Msg_type | Msg_text |
+---------+--------------+----------+----------+
| test.t1 | preload_keys | status   | OK       |
| test.t2 | preload_keys | status   | OK       |
+---------+--------------+----------+----------+
</pre><p>
          The <code class="literal">IGNORE LEAVES</code> modifier causes only
          blocks for the nonleaf nodes of the index to be preloaded.
          Thus, the statement shown preloads all index blocks from
          <code class="literal">t1</code>, but only blocks for the nonleaf nodes
          from <code class="literal">t2</code>.
        </p><p>
          If an index has been assigned to a key cache using a
          <a class="link" href="sql-statements.html#cache-index" title="13.7.8.2 CACHE INDEX Statement"><code class="literal">CACHE INDEX</code></a> statement,
          preloading places index blocks into that cache. Otherwise, the
          index is loaded into the default key cache.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="key-cache-block-size"></a>8.10.2.5 Key Cache Block Size</h4>

</div>

</div>

</div>
<p>
          It is possible to specify the size of the block buffers for an
          individual key cache using the
          <a class="link" href="server-administration.html#sysvar_key_cache_block_size"><code class="literal">key_cache_block_size</code></a>
          variable. This permits tuning of the performance of I/O
          operations for index files.
        </p><p>
          The best performance for I/O operations is achieved when the
          size of read buffers is equal to the size of the native
          operating system I/O buffers. But setting the size of key
          nodes equal to the size of the I/O buffer does not always
          ensure the best overall performance. When reading the big leaf
          nodes, the server pulls in a lot of unnecessary data,
          effectively preventing reading other leaf nodes.
        </p><p>
          To control the size of blocks in the <code class="filename">.MYI</code>
          index file of <code class="literal">MyISAM</code> tables, use the
          <a class="link" href="server-administration.html#option_mysqld_myisam-block-size"><code class="option">--myisam-block-size</code></a> option at
          server startup.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="key-cache-restructuring"></a>8.10.2.6 Restructuring a Key Cache</h4>

</div>

</div>

</div>
<p>
          A key cache can be restructured at any time by updating its
          parameter values. For example:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SET GLOBAL cold_cache.key_buffer_size=4*1024*1024;</code></strong>
</pre><p>
          If you assign to either the
          <a class="link" href="server-administration.html#sysvar_key_buffer_size"><code class="literal">key_buffer_size</code></a> or
          <a class="link" href="server-administration.html#sysvar_key_cache_block_size"><code class="literal">key_cache_block_size</code></a> key
          cache component a value that differs from the component's
          current value, the server destroys the cache's old
          structure and creates a new one based on the new values. If
          the cache contains any dirty blocks, the server saves them to
          disk before destroying and re-creating the cache.
          Restructuring does not occur if you change other key cache
          parameters.
        </p><p>
          When restructuring a key cache, the server first flushes the
          contents of any dirty buffers to disk. After that, the cache
          contents become unavailable. However, restructuring does not
          block queries that need to use indexes assigned to the cache.
          Instead, the server directly accesses the table indexes using
          native file system caching. File system caching is not as
          efficient as using a key cache, so although queries execute, a
          slowdown can be anticipated. After the cache has been
          restructured, it becomes available again for caching indexes
          assigned to it, and the use of file system caching for the
          indexes ceases.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="statement-caching"></a>8.10.3 Caching of Prepared Statements and Stored Programs</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444346662320"></a><a class="indexterm" name="idm46444346660864"></a><p>
        For certain statements that a client might execute multiple
        times during a session, the server converts the statement to an
        internal structure and caches that structure to be used during
        execution. Caching enables the server to perform more
        efficiently because it avoids the overhead of reconverting the
        statement should it be needed again during the session.
        Conversion and caching occurs for these statements:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Prepared statements, both those processed at the SQL level
            (using the <a class="link" href="sql-statements.html#prepare" title="13.5.1 PREPARE Statement"><code class="literal">PREPARE</code></a> statement)
            and those processed using the binary client/server protocol
            (using the
            <a class="link" href="connectors-apis.html#mysql-stmt-prepare" title="28.7.10.21 mysql_stmt_prepare()"><code class="literal">mysql_stmt_prepare()</code></a> C API
            function). The
            <a class="link" href="server-administration.html#sysvar_max_prepared_stmt_count"><code class="literal">max_prepared_stmt_count</code></a>
            system variable controls the total number of statements the
            server caches. (The sum of the number of prepared statements
            across all sessions.)
          </p></li><li class="listitem"><p>
            Stored programs (stored procedures and functions, triggers,
            and events). In this case, the server converts and caches
            the entire program body. The
            <a class="link" href="server-administration.html#sysvar_stored_program_cache"><code class="literal">stored_program_cache</code></a> system
            variable indicates the approximate number of stored programs
            the server caches per session.
</p></li></ul>
</div>
<p>
        The server maintains caches for prepared statements and stored
        programs on a per-session basis. Statements cached for one
        session are not accessible to other sessions. When a session
        ends, the server discards any statements cached for it.
      </p><p>
        When the server uses a cached internal statement structure, it
        must take care that the structure does not go out of date.
        Metadata changes can occur for an object used by the statement,
        causing a mismatch between the current object definition and the
        definition as represented in the internal statement structure.
        Metadata changes occur for DDL statements such as those that
        create, drop, alter, rename, or truncate tables, or that
        analyze, optimize, or repair tables. Table content changes (for
        example, with <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a> or
        <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a>) do not change metadata,
        nor do <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements.
      </p><p>
        Here is an illustration of the problem. Suppose that a client
        prepares this statement:
      </p><pre data-lang="sql" class="programlisting">PREPARE s1 FROM 'SELECT * FROM t1';</pre><p>
        The <code class="literal">SELECT *</code> expands in the internal
        structure to the list of columns in the table. If the set of
        columns in the table is modified with <code class="literal">ALTER
        TABLE</code>, the prepared statement goes out of date. If the
        server does not detect this change the next time the client
        executes <code class="literal">s1</code>, the prepared statement will
        return incorrect results.
      </p><p>
        To avoid problems caused by metadata changes to tables or views
        referred to by the prepared statement, the server detects these
        changes and automatically reprepares the statement when it is
        next executed. That is, the server reparses the statement and
        rebuilds the internal structure. Reparsing also occurs after
        referenced tables or views are flushed from the table definition
        cache, either implicitly to make room for new entries in the
        cache, or explicitly due to <a class="link" href="sql-statements.html#flush-tables"><code class="literal">FLUSH
        TABLES</code></a>.
      </p><p>
        Similarly, if changes occur to objects used by a stored program,
        the server reparses affected statements within the program.
      </p><p>
        The server also detects metadata changes for objects in
        expressions. These might be used in statements specific to
        stored programs, such as <code class="literal">DECLARE CURSOR</code> or
        flow-control statements such as
        <a class="link" href="sql-statements.html#if" title="13.6.5.2 IF Statement"><code class="literal">IF</code></a>,
        <a class="link" href="sql-statements.html#case" title="13.6.5.1 CASE Statement"><code class="literal">CASE</code></a>, and
        <a class="link" href="sql-statements.html#return" title="13.6.5.7 RETURN Statement"><code class="literal">RETURN</code></a>.
      </p><p>
        To avoid reparsing entire stored programs, the server reparses
        affected statements or expressions within a program only as
        needed. Examples:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Suppose that metadata for a table or view is changed.
            Reparsing occurs for a <code class="literal">SELECT *</code> within
            the program that accesses the table or view, but not for a
            <code class="literal">SELECT *</code> that does not access the table
            or view.
          </p></li><li class="listitem"><p>
            When a statement is affected, the server reparses it only
            partially if possible. Consider this
            <a class="link" href="sql-statements.html#case" title="13.6.5.1 CASE Statement"><code class="literal">CASE</code></a> statement:
          </p><pre data-lang="sql" class="programlisting">CASE <em class="replaceable"><code>case_expr</code></em>
  WHEN <em class="replaceable"><code>when_expr1</code></em> ...
  WHEN <em class="replaceable"><code>when_expr2</code></em> ...
  WHEN <em class="replaceable"><code>when_expr3</code></em> ...
  ...
END CASE
</pre><p>
            If a metadata change affects only <code class="literal">WHEN
            <em class="replaceable"><code>when_expr3</code></em></code>, that
            expression is reparsed. <em class="replaceable"><code>case_expr</code></em>
            and the other <code class="literal">WHEN</code> expressions are not
            reparsed.
</p></li></ul>
</div>
<p>
        Reparsing uses the default database and SQL mode that were in
        effect for the original conversion to internal form.
      </p><p>
        The server attempts reparsing up to three times. An error occurs
        if all attempts fail.
      </p><p>
        Reparsing is automatic, but to the extent that it occurs,
        diminishes prepared statement and stored program performance.
      </p><p>
        For prepared statements, the
        <a class="link" href="server-administration.html#statvar_Com_xxx"><code class="literal">Com_stmt_reprepare</code></a>
        status variable tracks the number of repreparations.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a name="locking-issues"></a>8.11 Optimizing Locking Operations</h2>

</div>

</div>

</div>
<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#internal-locking">8.11.1 Internal Locking Methods</a></span></dt><dt><span class="section"><a href="optimization.html#table-locking">8.11.2 Table Locking Issues</a></span></dt><dt><span class="section"><a href="optimization.html#concurrent-inserts">8.11.3 Concurrent Inserts</a></span></dt><dt><span class="section"><a href="optimization.html#metadata-locking">8.11.4 Metadata Locking</a></span></dt><dt><span class="section"><a href="optimization.html#external-locking">8.11.5 External Locking</a></span></dt></dl>
</div>
<p>
      MySQL manages contention for table contents using
      <a class="link" href="glossary.html#glos_locking" title="locking">locking</a>:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
          Internal locking is performed within the MySQL server itself
          to manage contention for table contents by multiple threads.
          This type of locking is internal because it is performed
          entirely by the server and involves no other programs. See
          <a class="xref" href="optimization.html#internal-locking" title="8.11.1 Internal Locking Methods">Section 8.11.1, “Internal Locking Methods”</a>.
        </p></li><li class="listitem"><p>
          External locking occurs when the server and other programs
          lock <a class="link" href="storage-engines.html#myisam-storage-engine" title="16.2 The MyISAM Storage Engine"><code class="literal">MyISAM</code></a> table files to
          coordinate among themselves which program can access the
          tables at which time. See <a class="xref" href="optimization.html#external-locking" title="8.11.5 External Locking">Section 8.11.5, “External Locking”</a>.
</p></li></ul>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="internal-locking"></a>8.11.1 Internal Locking Methods</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444346609488"></a><a class="indexterm" name="idm46444346608448"></a><a class="indexterm" name="idm46444346606960"></a><a class="indexterm" name="idm46444346605888"></a><a class="indexterm" name="idm46444346604400"></a><a class="indexterm" name="idm46444346603328"></a><a class="indexterm" name="idm46444346602256"></a><a class="indexterm" name="idm46444346600768"></a><a class="indexterm" name="idm46444346599280"></a><p>
        This section discusses internal locking; that is, locking
        performed within the MySQL server itself to manage contention
        for table contents by multiple sessions. This type of locking is
        internal because it is performed entirely by the server and
        involves no other programs. For locking performed on MySQL files
        by other programs, see <a class="xref" href="optimization.html#external-locking" title="8.11.5 External Locking">Section 8.11.5, “External Locking”</a>.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a class="xref" href="optimization.html#internal-row-level-locking" title="Row-Level Locking">Row-Level Locking</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#internal-table-level-locking" title="Table-Level Locking">Table-Level Locking</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#internal-locking-choices" title="Choosing the Type of Locking">Choosing the Type of Locking</a></p></li></ul>
</div>

<div class="simplesect">

<div class="titlepage">
<div>

<div class="simple">
<h4 class="title"><a name="internal-row-level-locking"></a>Row-Level Locking</h4>

</div>

</div>

</div>
<p>
          MySQL uses <a class="link" href="glossary.html#glos_row_lock" title="row lock">row-level
          locking</a> for <code class="literal">InnoDB</code> tables to support
          simultaneous write access by multiple sessions, making them
          suitable for multi-user, highly concurrent, and OLTP
          applications.
        </p><p>
          To avoid <a class="link" href="glossary.html#glos_deadlock" title="deadlock">deadlocks</a> when
          performing multiple concurrent write operations on a single
          <code class="literal">InnoDB</code> table, acquire necessary locks at
          the start of the transaction by issuing a <code class="literal">SELECT ...
          FOR UPDATE</code> statement for each group of rows expected
          to be modified, even if the data change statements come later
          in the transaction. If transactions modify or lock more than
          one table, issue the applicable statements in the same order
          within each transaction. Deadlocks affect performance rather
          than representing a serious error, because
          <code class="literal">InnoDB</code> automatically
          <a class="link" href="glossary.html#glos_deadlock_detection" title="deadlock detection">detects</a>
          deadlock conditions by default and rolls back one of the
          affected transactions.
        </p><p>
          On high concurrency systems, deadlock detection can cause a
          slowdown when numerous threads wait for the same lock. At
          times, it may be more efficient to disable deadlock detection
          and rely on the
          <a class="link" href="innodb-storage-engine.html#sysvar_innodb_lock_wait_timeout"><code class="literal">innodb_lock_wait_timeout</code></a>
          setting for transaction rollback when a deadlock occurs.
          Deadlock detection can be disabled using the
          <a class="link" href="innodb-storage-engine.html#sysvar_innodb_deadlock_detect"><code class="literal">innodb_deadlock_detect</code></a>
          configuration option.
        </p><p>
          Advantages of row-level locking:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Fewer lock conflicts when different sessions access
              different rows.
            </p></li><li class="listitem"><p>
              Fewer changes for rollbacks.
            </p></li><li class="listitem"><p>
              Possible to lock a single row for a long time.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="internal-table-level-locking"></a>Table-Level Locking</h4>

</div>

</div>

</div>
<p>
          MySQL uses <a class="link" href="glossary.html#glos_table_lock" title="table lock">table-level
          locking</a> for <code class="literal">MyISAM</code>,
          <code class="literal">MEMORY</code>, and <code class="literal">MERGE</code>
          tables, permitting only one session to update those tables at
          a time. This locking level makes these storage engines more
          suitable for read-only, read-mostly, or single-user
          applications.
        </p><p>
          These storage engines avoid
          <a class="link" href="glossary.html#glos_deadlock" title="deadlock">deadlocks</a> by always
          requesting all needed locks at once at the beginning of a
          query and always locking the tables in the same order. The
          tradeoff is that this strategy reduces concurrency; other
          sessions that want to modify the table must wait until the
          current data change statement finishes.
        </p><p>
          Advantages of table-level locking:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Relatively little memory required (row locking requires
              memory per row or group of rows locked)
            </p></li><li class="listitem"><p>
              Fast when used on a large part of the table because only a
              single lock is involved.
            </p></li><li class="listitem"><p>
              Fast if you often do <code class="literal">GROUP BY</code>
              operations on a large part of the data or must scan the
              entire table frequently.
</p></li></ul>
</div>
<p>
          MySQL grants table write locks as follows:
</p>
<div class="orderedlist">
<ol class="orderedlist" type="1"><li class="listitem"><p>
              If there are no locks on the table, put a write lock on
              it.
            </p></li><li class="listitem"><p>
              Otherwise, put the lock request in the write lock queue.
</p></li></ol>
</div>
<p>
          MySQL grants table read locks as follows:
</p>
<div class="orderedlist">
<ol class="orderedlist" type="1"><li class="listitem"><p>
              If there are no write locks on the table, put a read lock
              on it.
            </p></li><li class="listitem"><p>
              Otherwise, put the lock request in the read lock queue.
</p></li></ol>
</div>
<p>
          Table updates are given higher priority than table retrievals.
          Therefore, when a lock is released, the lock is made available
          to the requests in the write lock queue and then to the
          requests in the read lock queue. This ensures that updates to
          a table are not <span class="quote">“<span class="quote">starved</span>”</span> even when there is
          heavy <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> activity for the
          table. However, if there are many updates for a table,
          <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements wait until
          there are no more updates.
        </p><p>
          For information on altering the priority of reads and writes,
          see <a class="xref" href="optimization.html#table-locking" title="8.11.2 Table Locking Issues">Section 8.11.2, “Table Locking Issues”</a>.
        </p><p>
          You can analyze the table lock contention on your system by
          checking the
          <a class="link" href="server-administration.html#statvar_Table_locks_immediate"><code class="literal">Table_locks_immediate</code></a> and
          <a class="link" href="server-administration.html#statvar_Table_locks_waited"><code class="literal">Table_locks_waited</code></a> status
          variables, which indicate the number of times that requests
          for table locks could be granted immediately and the number
          that had to wait, respectively:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SHOW STATUS LIKE 'Table%';</code></strong>
+-----------------------+---------+
| Variable_name         | Value   |
+-----------------------+---------+
| Table_locks_immediate | 1151552 |
| Table_locks_waited    | 15324   |
+-----------------------+---------+
</pre><p>
          The Performance Schema lock tables also provide locking
          information. See
          <a class="xref" href="performance-schema.html#performance-schema-lock-tables" title="26.12.13 Performance Schema Lock Tables">Section 26.12.13, “Performance Schema Lock Tables”</a>.
        </p><a class="indexterm" name="idm46444346549264"></a><a class="indexterm" name="idm46444346548192"></a><p>
          The <code class="literal">MyISAM</code> storage engine supports
          concurrent inserts to reduce contention between readers and
          writers for a given table: If a <code class="literal">MyISAM</code>
          table has no free blocks in the middle of the data file, rows
          are always inserted at the end of the data file. In this case,
          you can freely mix concurrent
          <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a> and
          <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements for a
          <code class="literal">MyISAM</code> table without locks. That is, you
          can insert rows into a <code class="literal">MyISAM</code> table at the
          same time other clients are reading from it. Holes can result
          from rows having been deleted from or updated in the middle of
          the table. If there are holes, concurrent inserts are disabled
          but are enabled again automatically when all holes have been
          filled with new data. To control this behavior, use the
          <a class="link" href="server-administration.html#sysvar_concurrent_insert"><code class="literal">concurrent_insert</code></a> system
          variable. See <a class="xref" href="optimization.html#concurrent-inserts" title="8.11.3 Concurrent Inserts">Section 8.11.3, “Concurrent Inserts”</a>.
        </p><p>
          If you acquire a table lock explicitly with
          <a class="link" href="sql-statements.html#lock-tables" title="13.3.6 LOCK TABLES and UNLOCK TABLES Statements"><code class="literal">LOCK TABLES</code></a>, you can request a
          <code class="literal">READ LOCAL</code> lock rather than a
          <code class="literal">READ</code> lock to enable other sessions to
          perform concurrent inserts while you have the table locked.
        </p><p>
          To perform many <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a> and
          <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> operations on a table
          <code class="literal">t1</code> when concurrent inserts are not
          possible, you can insert rows into a temporary table
          <code class="literal">temp_t1</code> and update the real table with the
          rows from the temporary table:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>LOCK TABLES t1 WRITE, temp_t1 WRITE;</code></strong>
mysql&gt; <strong class="userinput"><code>INSERT INTO t1 SELECT * FROM temp_t1;</code></strong>
mysql&gt; <strong class="userinput"><code>DELETE FROM temp_t1;</code></strong>
mysql&gt; <strong class="userinput"><code>UNLOCK TABLES;</code></strong>
</pre>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="internal-locking-choices"></a>Choosing the Type of Locking</h4>

</div>

</div>

</div>
<p>
          Generally, table locks are superior to row-level locks in the
          following cases:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Most statements for the table are reads.
            </p></li><li class="listitem"><p>
              Statements for the table are a mix of reads and writes,
              where writes are updates or deletes for a single row that
              can be fetched with one key read:
            </p><pre data-lang="sql" class="programlisting">UPDATE <em class="replaceable"><code>tbl_name</code></em> SET <em class="replaceable"><code>column</code></em>=<em class="replaceable"><code>value</code></em> WHERE <em class="replaceable"><code>unique_key_col</code></em>=<em class="replaceable"><code>key_value</code></em>;
DELETE FROM <em class="replaceable"><code>tbl_name</code></em> WHERE <em class="replaceable"><code>unique_key_col</code></em>=<em class="replaceable"><code>key_value</code></em>;
</pre></li><li class="listitem"><p>
              <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> combined with
              concurrent <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>
              statements, and very few
              <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a> or
              <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a> statements.
            </p></li><li class="listitem"><p>
              Many scans or <code class="literal">GROUP BY</code> operations on
              the entire table without any writers.
</p></li></ul>
</div>
<p>
          With higher-level locks, you can more easily tune applications
          by supporting locks of different types, because the lock
          overhead is less than for row-level locks.
        </p><p>
          Options other than row-level locking:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Versioning (such as that used in MySQL for concurrent
              inserts) where it is possible to have one writer at the
              same time as many readers. This means that the database or
              table supports different views for the data depending on
              when access begins. Other common terms for this are
              <span class="quote">“<span class="quote">time travel,</span>”</span> <span class="quote">“<span class="quote">copy on write,</span>”</span>
              or <span class="quote">“<span class="quote">copy on demand.</span>”</span>
            </p></li><li class="listitem"><p>
              Copy on demand is in many cases superior to row-level
              locking. However, in the worst case, it can use much more
              memory than using normal locks.
            </p></li><li class="listitem"><p>
              Instead of using row-level locks, you can employ
              application-level locks, such as those provided by
              <a class="link" href="functions.html#function_get-lock"><code class="literal">GET_LOCK()</code></a> and
              <a class="link" href="functions.html#function_release-lock"><code class="literal">RELEASE_LOCK()</code></a> in MySQL.
              These are advisory locks, so they work only with
              applications that cooperate with each other. See
              <a class="xref" href="functions.html#locking-functions" title="12.14 Locking Functions">Section 12.14, “Locking Functions”</a>.
</p></li></ul>
</div>

</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="table-locking"></a>8.11.2 Table Locking Issues</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444346500144"></a><p>
        <code class="literal">InnoDB</code> tables use row-level locking so that
        multiple sessions and applications can read from and write to
        the same table simultaneously, without making each other wait or
        producing inconsistent results. For this storage engine, avoid
        using the <a class="link" href="sql-statements.html#lock-tables" title="13.3.6 LOCK TABLES and UNLOCK TABLES Statements"><code class="literal">LOCK TABLES</code></a> statement,
        because it does not offer any extra protection, but instead
        reduces concurrency. The automatic row-level locking makes these
        tables suitable for your busiest databases with your most
        important data, while also simplifying application logic since
        you do not need to lock and unlock tables. Consequently, the
        <code class="literal">InnoDB</code> storage engine is the default in
        MySQL.
      </p><p>
        MySQL uses table locking (instead of page, row, or column
        locking) for all storage engines except
        <code class="literal">InnoDB</code>. The locking operations themselves do
        not have much overhead. But because only one session can write
        to a table at any one time, for best performance with these
        other storage engines, use them primarily for tables that are
        queried often and rarely inserted into or updated.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a class="xref" href="optimization.html#table-locking-innodb" title="Performance Considerations Favoring InnoDB">Performance Considerations Favoring InnoDB</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#table-locking-workarounds" title="Workarounds for Locking Performance Issues">Workarounds for Locking Performance Issues</a></p></li></ul>
</div>

<div class="simplesect">

<div class="titlepage">
<div>

<div class="simple">
<h4 class="title"><a name="table-locking-innodb"></a>Performance Considerations Favoring InnoDB</h4>

</div>

</div>

</div>
<p>
          When choosing whether to create a table using
          <code class="literal">InnoDB</code> or a different storage engine, keep
          in mind the following disadvantages of table locking:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Table locking enables many sessions to read from a table
              at the same time, but if a session wants to write to a
              table, it must first get exclusive access, meaning it
              might have to wait for other sessions to finish with the
              table first. During the update, all other sessions that
              want to access this particular table must wait until the
              update is done.
            </p></li><li class="listitem"><p>
              Table locking causes problems when a session is waiting
              because the disk is full and free space needs to become
              available before the session can proceed. In this case,
              all sessions that want to access the problem table are
              also put in a waiting state until more disk space is made
              available.
            </p></li><li class="listitem"><p>
              A <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statement that
              takes a long time to run prevents other sessions from
              updating the table in the meantime, making the other
              sessions appear slow or unresponsive. While a session is
              waiting to get exclusive access to the table for updates,
              other sessions that issue
              <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements will
              queue up behind it, reducing concurrency even for
              read-only sessions.
</p></li></ul>
</div>

</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="table-locking-workarounds"></a>Workarounds for Locking Performance Issues</h4>

</div>

</div>

</div>
<p>
          The following items describe some ways to avoid or reduce
          contention caused by table locking:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              Consider switching the table to the
              <code class="literal">InnoDB</code> storage engine, either using
              <code class="literal">CREATE TABLE ... ENGINE=INNODB</code> during
              setup, or using <code class="literal">ALTER TABLE ...
              ENGINE=INNODB</code> for an existing table. See
              <a class="xref" href="innodb-storage-engine.html" title="Chapter 15 The InnoDB Storage Engine">Chapter 15, <i>The InnoDB Storage Engine</i></a> for more details
              about this storage engine.
            </p></li><li class="listitem"><p>
              Optimize <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements
              to run faster so that they lock tables for a shorter time.
              You might have to create some summary tables to do this.
            </p></li><li class="listitem"><p>
              Start <a class="link" href="programs.html#mysqld" title="4.3.1 mysqld — The MySQL Server"><span class="command"><strong>mysqld</strong></span></a> with
              <a class="link" href="server-administration.html#sysvar_low_priority_updates"><code class="option">--low-priority-updates</code></a>. For
              storage engines that use only table-level locking (such as
              <code class="literal">MyISAM</code>, <code class="literal">MEMORY</code>, and
              <code class="literal">MERGE</code>), this gives all statements that
              update (modify) a table lower priority than
              <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements. In this
              case, the second <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>
              statement in the preceding scenario would execute before
              the <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a> statement, and
              would not wait for the first
              <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> to finish.
            </p></li><li class="listitem"><p>
              To specify that all updates issued in a specific
              connection should be done with low priority, set the
              <a class="link" href="server-administration.html#sysvar_low_priority_updates"><code class="literal">low_priority_updates</code></a>
              server system variable equal to 1.
            </p></li><li class="listitem"><p>
              To give a specific <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>,
              <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a>, or
              <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a> statement lower
              priority, use the <code class="literal">LOW_PRIORITY</code>
              attribute.
            </p></li><li class="listitem"><p>
              To give a specific <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>
              statement higher priority, use the
              <code class="literal">HIGH_PRIORITY</code> attribute. See
              <a class="xref" href="sql-statements.html#select" title="13.2.10 SELECT Statement">Section 13.2.10, “SELECT Statement”</a>.
            </p></li><li class="listitem"><p>
              Start <a class="link" href="programs.html#mysqld" title="4.3.1 mysqld — The MySQL Server"><span class="command"><strong>mysqld</strong></span></a> with a low value for the
              <a class="link" href="server-administration.html#sysvar_max_write_lock_count"><code class="literal">max_write_lock_count</code></a>
              system variable to force MySQL to temporarily elevate the
              priority of all <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>
              statements that are waiting for a table after a specific
              number of inserts to the table occur. This permits
              <code class="literal">READ</code> locks after a certain number of
              <code class="literal">WRITE</code> locks.
            </p></li><li class="listitem"><p>
              If you have problems with mixed
              <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> and
              <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a> statements, the
              <code class="literal">LIMIT</code> option to
              <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a> may help. See
              <a class="xref" href="sql-statements.html#delete" title="13.2.2 DELETE Statement">Section 13.2.2, “DELETE Statement”</a>.
            </p></li><li class="listitem"><p>
              Using <code class="literal">SQL_BUFFER_RESULT</code> with
              <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements can help
              to make the duration of table locks shorter. See
              <a class="xref" href="sql-statements.html#select" title="13.2.10 SELECT Statement">Section 13.2.10, “SELECT Statement”</a>.
            </p></li><li class="listitem"><p>
              Splitting table contents into separate tables may help, by
              allowing queries to run against columns in one table,
              while updates are confined to columns in a different
              table.
            </p></li><li class="listitem"><p>
              You could change the locking code in
              <code class="filename">mysys/thr_lock.c</code> to use a single
              queue. In this case, write locks and read locks would have
              the same priority, which might help some applications.
</p></li></ul>
</div>

</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="concurrent-inserts"></a>8.11.3 Concurrent Inserts</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444346432592"></a><a class="indexterm" name="idm46444346431520"></a><p>
        The <code class="literal">MyISAM</code> storage engine supports concurrent
        inserts to reduce contention between readers and writers for a
        given table: If a <code class="literal">MyISAM</code> table has no holes
        in the data file (deleted rows in the middle), an
        <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a> statement can be executed
        to add rows to the end of the table at the same time that
        <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements are reading
        rows from the table. If there are multiple
        <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a> statements, they are
        queued and performed in sequence, concurrently with the
        <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statements. The results of
        a concurrent <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a> may not be
        visible immediately.
      </p><p>
        The <a class="link" href="server-administration.html#sysvar_concurrent_insert"><code class="literal">concurrent_insert</code></a> system
        variable can be set to modify the concurrent-insert processing.
        By default, the variable is set to <code class="literal">AUTO</code> (or
        1) and concurrent inserts are handled as just described. If
        <a class="link" href="server-administration.html#sysvar_concurrent_insert"><code class="literal">concurrent_insert</code></a> is set to
        <code class="literal">NEVER</code> (or 0), concurrent inserts are
        disabled. If the variable is set to <code class="literal">ALWAYS</code>
        (or 2), concurrent inserts at the end of the table are permitted
        even for tables that have deleted rows. See also the description
        of the <a class="link" href="server-administration.html#sysvar_concurrent_insert"><code class="literal">concurrent_insert</code></a> system
        variable.
      </p><p>
        If you are using the binary log, concurrent inserts are
        converted to normal inserts for <code class="literal">CREATE ...
        SELECT</code> or
        <a class="link" href="sql-statements.html#insert-select" title="13.2.6.1 INSERT ... SELECT Statement"><code class="literal">INSERT ...
        SELECT</code></a> statements. This is done to ensure that you can
        re-create an exact copy of your tables by applying the log
        during a backup operation. See <a class="xref" href="server-administration.html#binary-log" title="5.4.4 The Binary Log">Section 5.4.4, “The Binary Log”</a>. In
        addition, for those statements a read lock is placed on the
        selected-from table such that inserts into that table are
        blocked. The effect is that concurrent inserts for that table
        must wait as well.
      </p><p>
        With <a class="link" href="sql-statements.html#load-data" title="13.2.7 LOAD DATA Statement"><code class="literal">LOAD DATA</code></a>, if you specify
        <code class="literal">CONCURRENT</code> with a <code class="literal">MyISAM</code>
        table that satisfies the condition for concurrent inserts (that
        is, it contains no free blocks in the middle), other sessions
        can retrieve data from the table while <a class="link" href="sql-statements.html#load-data" title="13.2.7 LOAD DATA Statement"><code class="literal">LOAD
        DATA</code></a> is executing. Use of the
        <code class="literal">CONCURRENT</code> option affects the performance of
        <a class="link" href="sql-statements.html#load-data" title="13.2.7 LOAD DATA Statement"><code class="literal">LOAD DATA</code></a> a bit, even if no other
        session is using the table at the same time.
      </p><p>
        If you specify <code class="literal">HIGH_PRIORITY</code>, it overrides
        the effect of the
        <a class="link" href="server-administration.html#sysvar_low_priority_updates"><code class="option">--low-priority-updates</code></a> option if
        the server was started with that option. It also causes
        concurrent inserts not to be used.
      </p><p>
        For <a class="link" href="sql-statements.html#lock-tables" title="13.3.6 LOCK TABLES and UNLOCK TABLES Statements"><code class="literal">LOCK
        TABLE</code></a>, the difference between <code class="literal">READ
        LOCAL</code> and <code class="literal">READ</code> is that
        <code class="literal">READ LOCAL</code> permits nonconflicting
        <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a> statements (concurrent
        inserts) to execute while the lock is held. However, this cannot
        be used if you are going to manipulate the database using
        processes external to the server while you hold the lock.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="metadata-locking"></a>8.11.4 Metadata Locking</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444346395856"></a><a class="indexterm" name="idm46444346394784"></a><a class="indexterm" name="idm46444346393296"></a><p>
        MySQL uses metadata locking to manage concurrent access to
        database objects and to ensure data consistency. Metadata
        locking applies not just to tables, but also to schemas, stored
        programs (procedures, functions, triggers, scheduled events),
        tablespaces, user locks acquired with the
        <a class="link" href="functions.html#function_get-lock"><code class="literal">GET_LOCK()</code></a> function (see
        <a class="xref" href="functions.html#locking-functions" title="12.14 Locking Functions">Section 12.14, “Locking Functions”</a>), and locks acquired with
        the locking service described in
        <a class="xref" href="extending-mysql.html#locking-service" title="29.3.1 The Locking Service">Section 29.3.1, “The Locking Service”</a>.
      </p><p>
        The Performance Schema
        <a class="link" href="performance-schema.html#metadata-locks-table" title="26.12.13.3 The metadata_locks Table"><code class="literal">metadata_locks</code></a> table exposes
        metadata lock information, which can be useful for seeing which
        sessions hold locks, are blocked waiting for locks, and so
        forth. For details, see <a class="xref" href="performance-schema.html#metadata-locks-table" title="26.12.13.3 The metadata_locks Table">Section 26.12.13.3, “The metadata_locks Table”</a>.
      </p><p>
        Metadata locking does involve some overhead, which increases as
        query volume increases. Metadata contention increases the more
        that multiple queries attempt to access the same objects.
      </p><p>
        Metadata locking is not a replacement for the table definition
        cache, and its mutexes and locks differ from the
        <code class="literal">LOCK_open</code> mutex. The following discussion
        provides some information about how metadata locking works.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p><a class="xref" href="optimization.html#metadata-lock-acquisition" title="Metadata Lock Acquisition">Metadata Lock Acquisition</a></p></li><li class="listitem"><p><a class="xref" href="optimization.html#metadata-lock-release" title="Metadata Lock Release">Metadata Lock Release</a></p></li></ul>
</div>

<div class="simplesect">

<div class="titlepage">
<div>

<div class="simple">
<h4 class="title"><a name="metadata-lock-acquisition"></a>Metadata Lock Acquisition</h4>

</div>

</div>

</div>
<p>
          If there are multiple waiters for a given lock, the
          highest-priority lock request is satisfied first, with an
          exception related to the
          <a class="link" href="server-administration.html#sysvar_max_write_lock_count"><code class="literal">max_write_lock_count</code></a> system
          variable. Write lock requests have higher priority than read
          lock requests. However, if
          <a class="link" href="server-administration.html#sysvar_max_write_lock_count"><code class="literal">max_write_lock_count</code></a> is set
          to some low value (say, 10), read lock requests may be
          preferred over pending write lock requests if the read lock
          requests have already been passed over in favor of 10 write
          lock requests. Normally this behavior does not occur because
          <a class="link" href="server-administration.html#sysvar_max_write_lock_count"><code class="literal">max_write_lock_count</code></a> by
          default has a very large value.
        </p><p>
          Statements acquire metadata locks one by one, not
          simultaneously, and perform deadlock detection in the process.
        </p><p>
          DML statements normally acquire locks in the order in which
          tables are mentioned in the statement.
        </p><p>
          DDL statements, <a class="link" href="sql-statements.html#lock-tables" title="13.3.6 LOCK TABLES and UNLOCK TABLES Statements"><code class="literal">LOCK TABLES</code></a>,
          and other similar statements try to reduce the number of
          possible deadlocks between concurrent DDL statements by
          acquiring locks on explicitly named tables in name order.
          Locks might be acquired in a different order for implicitly
          used tables (such as tables in foreign key relationships that
          also must be locked).
        </p><p>
          For example, <a class="link" href="sql-statements.html#rename-table" title="13.1.36 RENAME TABLE Statement"><code class="literal">RENAME TABLE</code></a> is a
          DDL statement that acquires locks in name order:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              This <a class="link" href="sql-statements.html#rename-table" title="13.1.36 RENAME TABLE Statement"><code class="literal">RENAME TABLE</code></a> statement
              renames <code class="literal">tbla</code> to something else, and
              renames <code class="literal">tblc</code> to
              <code class="literal">tbla</code>:
            </p><pre data-lang="sql" class="programlisting">RENAME TABLE tbla TO tbld, tblc TO tbla;</pre><p>
              The statement acquires metadata locks, in order, on
              <code class="literal">tbla</code>, <code class="literal">tblc</code>, and
              <code class="literal">tbld</code> (because <code class="literal">tbld</code>
              follows <code class="literal">tblc</code> in name order):
            </p></li><li class="listitem"><p>
              This slightly different statement also renames
              <code class="literal">tbla</code> to something else, and renames
              <code class="literal">tblc</code> to <code class="literal">tbla</code>:
            </p><pre data-lang="sql" class="programlisting">RENAME TABLE tbla TO tblb, tblc TO tbla;</pre><p>
              In this case, the statement acquires metadata locks, in
              order, on <code class="literal">tbla</code>,
              <code class="literal">tblb</code>, and <code class="literal">tblc</code>
              (because <code class="literal">tblb</code> precedes
              <code class="literal">tblc</code> in name order):
</p></li></ul>
</div>
<p>
          Both statements acquire locks on <code class="literal">tbla</code> and
          <code class="literal">tblc</code>, in that order, but differ in whether
          the lock on the remaining table name is acquired before or
          after <code class="literal">tblc</code>.
        </p><p>
          Metadata lock acquisition order can make a difference in
          operation outcome when multiple transactions execute
          concurrently, as the following example illustrates.
        </p><p>
          Begin with two tables <code class="literal">x</code> and
          <code class="literal">x_new</code> that have identical structure. Three
          clients issue statements that involve these tables:
        </p><p>
          Client 1:
        </p><pre data-lang="sql" class="programlisting">LOCK TABLE x WRITE, x_new WRITE;</pre><p>
          The statement requests and acquires write locks in name order
          on <code class="literal">x</code> and <code class="literal">x_new</code>.
        </p><p>
          Client 2:
        </p><pre data-lang="sql" class="programlisting">INSERT INTO x VALUES(1);</pre><p>
          The statement requests and blocks waiting for a write lock on
          <code class="literal">x</code>.
        </p><p>
          Client 3:
        </p><pre data-lang="sql" class="programlisting">RENAME TABLE x TO x_old, x_new TO x;</pre><p>
          The statement requests exclusive locks in name order on
          <code class="literal">x</code>, <code class="literal">x_new</code>, and
          <code class="literal">x_old</code>, but blocks waiting for the lock on
          <code class="literal">x</code>.
        </p><p>
          Client 1:
        </p><pre data-lang="sql" class="programlisting">UNLOCK TABLES;</pre><p>
          The statement releases the write locks on <code class="literal">x</code>
          and <code class="literal">x_new</code>. The exclusive lock request for
          <code class="literal">x</code> by Client 3 has higher priority than the
          write lock request by Client 2, so Client 3 acquires its lock
          on <code class="literal">x</code>, then also on <code class="literal">x_new</code>
          and <code class="literal">x_old</code>, performs the renaming, and
          releases its locks. Client 2 then acquires its lock on
          <code class="literal">x</code>, performs the insert, and releases its
          lock.
        </p><p>
          Lock acquisition order results in the
          <a class="link" href="sql-statements.html#rename-table" title="13.1.36 RENAME TABLE Statement"><code class="literal">RENAME TABLE</code></a> executing before
          the <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>. The
          <code class="literal">x</code> into which the insert occurs is the table
          that was named <code class="literal">x_new</code> when Client 2 issued
          the insert and was renamed to <code class="literal">x</code> by Client
          3:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT * FROM x;</code></strong>
+------+
| i    |
+------+
|    1 |
+------+

mysql&gt; <strong class="userinput"><code>SELECT * FROM x_old;</code></strong>
Empty set (0.01 sec)
</pre><p>
          Now begin instead with tables named <code class="literal">x</code> and
          <code class="literal">new_x</code> that have identical structure. Again,
          three clients issue statements that involve these tables:
        </p><p>
          Client 1:
        </p><pre data-lang="sql" class="programlisting">LOCK TABLE x WRITE, new_x WRITE;</pre><p>
          The statement requests and acquires write locks in name order
          on <code class="literal">new_x</code> and <code class="literal">x</code>.
        </p><p>
          Client 2:
        </p><pre data-lang="sql" class="programlisting">INSERT INTO x VALUES(1);</pre><p>
          The statement requests and blocks waiting for a write lock on
          <code class="literal">x</code>.
        </p><p>
          Client 3:
        </p><pre data-lang="sql" class="programlisting">RENAME TABLE x TO old_x, new_x TO x;</pre><p>
          The statement requests exclusive locks in name order on
          <code class="literal">new_x</code>, <code class="literal">old_x</code>, and
          <code class="literal">x</code>, but blocks waiting for the lock on
          <code class="literal">new_x</code>.
        </p><p>
          Client 1:
        </p><pre data-lang="sql" class="programlisting">UNLOCK TABLES;</pre><p>
          The statement releases the write locks on <code class="literal">x</code>
          and <code class="literal">new_x</code>. For <code class="literal">x</code>, the
          only pending request is by Client 2, so Client 2 acquires its
          lock, performs the insert, and releases the lock. For
          <code class="literal">new_x</code>, the only pending request is by
          Client 3, which is permitted to acquire that lock (and also
          the lock on <code class="literal">old_x</code>). The rename operation
          still blocks for the lock on <code class="literal">x</code> until the
          Client 2 insert finishes and releases its lock. Then Client 3
          acquires the lock on <code class="literal">x</code>, performs the
          rename, and releases its lock.
        </p><p>
          In this case, lock acquisition order results in the
          <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a> executing before the
          <a class="link" href="sql-statements.html#rename-table" title="13.1.36 RENAME TABLE Statement"><code class="literal">RENAME TABLE</code></a>. The
          <code class="literal">x</code> into which the insert occurs is the
          original <code class="literal">x</code>, now renamed to
          <code class="literal">old_x</code> by the rename operation:
        </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT * FROM x;</code></strong>
Empty set (0.01 sec)

mysql&gt; <strong class="userinput"><code>SELECT * FROM old_x;</code></strong>
+------+
| i    |
+------+
|    1 |
+------+
</pre><p>
          If order of lock acquisition in concurrent statements makes a
          difference to an application in operation outcome, as in the
          preceding example, you may be able to adjust the table names
          to affect the order of lock acquisition.
        </p><a class="indexterm" name="idm46444346294160"></a><p>
          Metadata locks are extended, as necessary, to tables related
          by a foreign key constraint to prevent conflicting DML and DDL
          operations from executing concurrently on the related tables.
          When updating a parent table, a metadata lock is taken on the
          child table while updating foreign key metadata. Foreign key
          metadata is owned by the child table.
</p>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="metadata-lock-release"></a>Metadata Lock Release</h4>

</div>

</div>

</div>
<p>
          To ensure transaction serializability, the server must not
          permit one session to perform a data definition language (DDL)
          statement on a table that is used in an uncompleted explicitly
          or implicitly started transaction in another session. The
          server achieves this by acquiring metadata locks on tables
          used within a transaction and deferring release of those locks
          until the transaction ends. A metadata lock on a table
          prevents changes to the table's structure. This locking
          approach has the implication that a table that is being used
          by a transaction within one session cannot be used in DDL
          statements by other sessions until the transaction ends.
        </p><p>
          This principle applies not only to transactional tables, but
          also to nontransactional tables. Suppose that a session begins
          a transaction that uses transactional table
          <code class="literal">t</code> and nontransactional table
          <code class="literal">nt</code> as follows:
        </p><pre data-lang="sql" class="programlisting">START TRANSACTION;
SELECT * FROM t;
SELECT * FROM nt;</pre><p>
          The server holds metadata locks on both <code class="literal">t</code>
          and <code class="literal">nt</code> until the transaction ends. If
          another session attempts a DDL or write lock operation on
          either table, it blocks until metadata lock release at
          transaction end. For example, a second session blocks if it
          attempts any of these operations:
        </p><pre data-lang="sql" class="programlisting">DROP TABLE t;
ALTER TABLE t ...;
DROP TABLE nt;
ALTER TABLE nt ...;
LOCK TABLE t ... WRITE;</pre><p>
          The same behavior applies for The
          <a class="link" href="sql-statements.html#lock-tables" title="13.3.6 LOCK TABLES and UNLOCK TABLES Statements"><code class="literal">LOCK TABLES ...
          READ</code></a>. That is, explicitly or implicitly started
          transactions that update any table (transactional or
          nontransactional) will block and be blocked by <code class="literal">LOCK
          TABLES ... READ</code> for that table.
        </p><p>
          If the server acquires metadata locks for a statement that is
          syntactically valid but fails during execution, it does not
          release the locks early. Lock release is still deferred to the
          end of the transaction because the failed statement is written
          to the binary log and the locks protect log consistency.
        </p><p>
          In autocommit mode, each statement is in effect a complete
          transaction, so metadata locks acquired for the statement are
          held only to the end of the statement.
        </p><p>
          Metadata locks acquired during a
          <a class="link" href="sql-statements.html#prepare" title="13.5.1 PREPARE Statement"><code class="literal">PREPARE</code></a> statement are released
          once the statement has been prepared, even if preparation
          occurs within a multiple-statement transaction.
        </p><p>
          As of MySQL 8.0.13, for XA transactions in
          <code class="literal">PREPARED</code> state, metadata locks are
          maintained across client disconnects and server restarts,
          until an <a class="link" href="sql-statements.html#xa" title="13.3.8 XA Transactions"><code class="literal">XA
          COMMIT</code></a> or <a class="link" href="sql-statements.html#xa" title="13.3.8 XA Transactions"><code class="literal">XA
          ROLLBACK</code></a> is executed.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="external-locking"></a>8.11.5 External Locking</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444346273312"></a><a class="indexterm" name="idm46444346272240"></a><p>
        External locking is the use of file system locking to manage
        contention for <a class="link" href="storage-engines.html#myisam-storage-engine" title="16.2 The MyISAM Storage Engine"><code class="literal">MyISAM</code></a> database
        tables by multiple processes. External locking is used in
        situations where a single process such as the MySQL server
        cannot be assumed to be the only process that requires access to
        tables. Here are some examples:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            If you run multiple servers that use the same database
            directory (not recommended), each server must have external
            locking enabled.
          </p></li><li class="listitem"><p>
            If you use <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> to perform table
            maintenance operations on
            <a class="link" href="storage-engines.html#myisam-storage-engine" title="16.2 The MyISAM Storage Engine"><code class="literal">MyISAM</code></a> tables, you must either
            ensure that the server is not running, or that the server
            has external locking enabled so that it locks table files as
            necessary to coordinate with <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a>
            for access to the tables. The same is true for use of
            <a class="link" href="programs.html#myisampack" title="4.6.6 myisampack — Generate Compressed, Read-Only MyISAM Tables"><span class="command"><strong>myisampack</strong></span></a> to pack
            <a class="link" href="storage-engines.html#myisam-storage-engine" title="16.2 The MyISAM Storage Engine"><code class="literal">MyISAM</code></a> tables.
          </p><p>
            If the server is run with external locking enabled, you can
            use <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> at any time for read
            operations such a checking tables. In this case, if the
            server tries to update a table that
            <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> is using, the server will wait
            for <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> to finish before it
            continues.
          </p><p>
            If you use <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> for write operations
            such as repairing or optimizing tables, or if you use
            <a class="link" href="programs.html#myisampack" title="4.6.6 myisampack — Generate Compressed, Read-Only MyISAM Tables"><span class="command"><strong>myisampack</strong></span></a> to pack tables, you
            <span class="emphasis"><em>must</em></span> always ensure that the
            <a class="link" href="programs.html#mysqld" title="4.3.1 mysqld — The MySQL Server"><span class="command"><strong>mysqld</strong></span></a> server is not using the table. If
            you do not stop <a class="link" href="programs.html#mysqld" title="4.3.1 mysqld — The MySQL Server"><span class="command"><strong>mysqld</strong></span></a>, at least do a
            <a class="link" href="programs.html#mysqladmin" title="4.5.2 mysqladmin — Client for Administering a MySQL Server"><span class="command"><strong>mysqladmin flush-tables</strong></span></a> before you run
            <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a>. Your tables <span class="emphasis"><em>may
            become corrupted</em></span> if the server and
            <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> access the tables
            simultaneously.
</p></li></ul>
</div>
<p>
        With external locking in effect, each process that requires
        access to a table acquires a file system lock for the table
        files before proceeding to access the table. If all necessary
        locks cannot be acquired, the process is blocked from accessing
        the table until the locks can be obtained (after the process
        that currently holds the locks releases them).
      </p><p>
        External locking affects server performance because the server
        must sometimes wait for other processes before it can access
        tables.
      </p><p>
        External locking is unnecessary if you run a single server to
        access a given data directory (which is the usual case) and if
        no other programs such as <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> need to
        modify tables while the server is running. If you only
        <span class="emphasis"><em>read</em></span> tables with other programs, external
        locking is not required, although <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a>
        might report warnings if the server changes tables while
        <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> is reading them.
      </p><p>
        With external locking disabled, to use
        <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a>, you must either stop the server
        while <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> executes or else lock and
        flush the tables before running <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a>. To
        avoid this requirement, use the <a class="link" href="sql-statements.html#check-table" title="13.7.3.2 CHECK TABLE Statement"><code class="literal">CHECK
        TABLE</code></a> and <a class="link" href="sql-statements.html#repair-table" title="13.7.3.5 REPAIR TABLE Statement"><code class="literal">REPAIR TABLE</code></a>
        statements to check and repair
        <a class="link" href="storage-engines.html#myisam-storage-engine" title="16.2 The MyISAM Storage Engine"><code class="literal">MyISAM</code></a> tables.
      </p><p>
        For <a class="link" href="programs.html#mysqld" title="4.3.1 mysqld — The MySQL Server"><span class="command"><strong>mysqld</strong></span></a>, external locking is controlled by
        the value of the
        <a class="link" href="server-administration.html#sysvar_skip_external_locking"><code class="literal">skip_external_locking</code></a> system
        variable. When this variable is enabled, external locking is
        disabled, and vice versa. External locking is disabled by
        default.
      </p><p>
        Use of external locking can be controlled at server startup by
        using the <a class="link" href="server-administration.html#option_mysqld_external-locking"><code class="option">--external-locking</code></a> or
        <a class="link" href="server-administration.html#option_mysqld_external-locking"><code class="option">--skip-external-locking</code></a>
        option.
      </p><p>
        If you do use external locking option to enable updates to
        <a class="link" href="storage-engines.html#myisam-storage-engine" title="16.2 The MyISAM Storage Engine"><code class="literal">MyISAM</code></a> tables from many MySQL
        processes, do not start the server with the
        <a class="link" href="server-administration.html#sysvar_delay_key_write"><code class="literal">delay_key_write</code></a> system variable
        set to <code class="literal">ALL</code> or use the
        <code class="literal">DELAY_KEY_WRITE=1</code> table option for any shared
        tables. Otherwise, index corruption can occur.
      </p><p>
        The easiest way to satisfy this condition is to always use
        <a class="link" href="server-administration.html#option_mysqld_external-locking"><code class="option">--external-locking</code></a> together with
        <a class="link" href="server-administration.html#sysvar_delay_key_write"><code class="option">--delay-key-write=OFF</code></a>. (This is
        not done by default because in many setups it is useful to have
        a mixture of the preceding options.)
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a name="optimizing-server"></a>8.12 Optimizing the MySQL Server</h2>

</div>

</div>

</div>
<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#disk-issues">8.12.1 Optimizing Disk I/O</a></span></dt><dt><span class="section"><a href="optimization.html#symbolic-links">8.12.2 Using Symbolic Links</a></span></dt><dt><span class="section"><a href="optimization.html#optimizing-memory">8.12.3 Optimizing Memory Use</a></span></dt></dl>
</div>
<p>
      This section discusses optimization techniques for the database
      server, primarily dealing with system configuration rather than
      tuning SQL statements. The information in this section is
      appropriate for DBAs who want to ensure performance and
      scalability across the servers they manage; for developers
      constructing installation scripts that include setting up the
      database; and people running MySQL themselves for development,
      testing, and so on who want to maximize their own productivity.
</p>
<div class="section">

<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="disk-issues"></a>8.12.1 Optimizing Disk I/O</h3>
</div>
</div>
</div>
<a class="indexterm" name="idm46444346215200"></a><a class="indexterm" name="idm46444346214128"></a><a class="indexterm" name="idm46444346212640"></a>
<div class="itemizedlist">
<p>
          This section describes ways to configure storage devices when
          you can devote more and faster storage hardware to the
          database server. For information about optimizing an
          <code class="literal">InnoDB</code> configuration to improve I/O
          performance, see <a class="xref" href="optimization.html#optimizing-innodb-diskio" title="8.5.8 Optimizing InnoDB Disk I/O">Section 8.5.8, “Optimizing InnoDB Disk I/O”</a>.
        </p><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            Disk seeks are a huge performance bottleneck. This problem
            becomes more apparent when the amount of data starts to grow
            so large that effective caching becomes impossible. For
            large databases where you access data more or less randomly,
            you can be sure that you need at least one disk seek to read
            and a couple of disk seeks to write things. To minimize this
            problem, use disks with low seek times.
          </p></li><li class="listitem"><p>
            Increase the number of available disk spindles (and thereby
            reduce the seek overhead) by either symlinking files to
            different disks or striping the disks:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                Using symbolic links
              </p><p>
                This means that, for <code class="literal">MyISAM</code> tables,
                you symlink the index file and data files from their
                usual location in the data directory to another disk
                (that may also be striped). This makes both the seek and
                read times better, assuming that the disk is not used
                for other purposes as well. See
                <a class="xref" href="optimization.html#symbolic-links" title="8.12.2 Using Symbolic Links">Section 8.12.2, “Using Symbolic Links”</a>.
              </p><p>
                Symbolic links are not supported for use with
                <code class="literal">InnoDB</code> tables. However, it is
                possible to place <code class="literal">InnoDB</code> data and log
                files on different physical disks. For more information,
                see <a class="xref" href="optimization.html#optimizing-innodb-diskio" title="8.5.8 Optimizing InnoDB Disk I/O">Section 8.5.8, “Optimizing InnoDB Disk I/O”</a>.
              </p></li><li class="listitem"><p>
                <a class="indexterm" name="idm46444346199792"></a>

                Striping
              </p><p>
                Striping means that you have many disks and put the
                first block on the first disk, the second block on the
                second disk, and the <em class="replaceable"><code>N</code></em>-th
                block on the (<code class="literal"><em class="replaceable"><code>N</code></em> MOD
                <em class="replaceable"><code>number_of_disks</code></em></code>)
                disk, and so on. This means if your normal data size is
                less than the stripe size (or perfectly aligned), you
                get much better performance. Striping is very dependent
                on the operating system and the stripe size, so
                benchmark your application with different stripe sizes.
                See <a class="xref" href="optimization.html#custom-benchmarks" title="8.13.2 Using Your Own Benchmarks">Section 8.13.2, “Using Your Own Benchmarks”</a>.
              </p><p>
                The speed difference for striping is
                <span class="emphasis"><em>very</em></span> dependent on the parameters.
                Depending on how you set the striping parameters and
                number of disks, you may get differences measured in
                orders of magnitude. You have to choose to optimize for
                random or sequential access.
</p></li></ul>
</div>
</li><li class="listitem"><p>
            For reliability, you may want to use RAID 0+1 (striping plus
            mirroring), but in this case, you need 2 ×
            <em class="replaceable"><code>N</code></em> drives to hold
            <em class="replaceable"><code>N</code></em> drives of data. This is
            probably the best option if you have the money for it.
            However, you may also have to invest in some
            volume-management software to handle it efficiently.
          </p></li><li class="listitem"><p>
            A good option is to vary the RAID level according to how
            critical a type of data is. For example, store
            semi-important data that can be regenerated on a RAID 0
            disk, but store really important data such as host
            information and logs on a RAID 0+1 or RAID
            <em class="replaceable"><code>N</code></em> disk. RAID
            <em class="replaceable"><code>N</code></em> can be a problem if you have
            many writes, due to the time required to update the parity
            bits.
          </p></li><li class="listitem"><p>
            You can also set the parameters for the file system that the
            database uses:
          </p><p>
            If you do not need to know when files were last accessed
            (which is not really useful on a database server), you can
            mount your file systems with the <code class="option">-o noatime</code>
            option. That skips updates to the last access time in inodes
            on the file system, which avoids some disk seeks.
          </p><p>
            On many operating systems, you can set a file system to be
            updated asynchronously by mounting it with the <code class="option">-o
            async</code> option. If your computer is reasonably
            stable, this should give you better performance without
            sacrificing too much reliability. (This flag is on by
            default on Linux.)
</p></li></ul>
</div>
<div class="simplesect">
<div class="titlepage">
<div>
<div class="simple">
<h4 class="title"><a name="disk-issues-nfs"></a>Using NFS with MySQL</h4>

</div>

</div>

</div>
<p>
          You should be cautious when considering whether to use NFS
          with MySQL. Potential issues, which vary by operating system
          and NFS version, include the following:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              MySQL data and log files placed on NFS volumes becoming
              locked and unavailable for use. Locking issues may occur
              in cases where multiple instances of MySQL access the same
              data directory or where MySQL is shut down improperly, due
              to a power outage, for example. NFS version 4 addresses
              underlying locking issues with the introduction of
              advisory and lease-based locking. However, sharing a data
              directory among MySQL instances is not recommended.
            </p></li><li class="listitem"><p>
              Data inconsistencies introduced due to messages received
              out of order or lost network traffic. To avoid this issue,
              use TCP with <code class="literal">hard</code> and
              <code class="literal">intr</code> mount options.
            </p></li><li class="listitem"><p>
              Maximum file size limitations. NFS Version 2 clients can
              only access the lowest 2GB of a file (signed 32 bit
              offset). NFS Version 3 clients support larger files (up to
              64 bit offsets). The maximum supported file size also
              depends on the local file system of the NFS server.
</p></li></ul>
</div>
<p>
          Using NFS within a professional SAN environment or other
          storage system tends to offer greater reliability than using
          NFS outside of such an environment. However, NFS within a SAN
          environment may be slower than directly attached or
          bus-attached non-rotational storage.
        </p><p>
          If you choose to use NFS, NFS Version 4 or later is
          recommended, as is testing your NFS setup thoroughly before
          deploying into a production environment.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="symbolic-links"></a>8.12.2 Using Symbolic Links</h3>

</div>

</div>

</div>
<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#symbolic-links-to-databases">8.12.2.1 Using Symbolic Links for Databases on Unix</a></span></dt><dt><span class="section"><a href="optimization.html#symbolic-links-to-tables">8.12.2.2 Using Symbolic Links for MyISAM Tables on Unix</a></span></dt><dt><span class="section"><a href="optimization.html#windows-symbolic-links">8.12.2.3 Using Symbolic Links for Databases on Windows</a></span></dt></dl>
</div>
<a class="indexterm" name="idm46444346175936"></a><a class="indexterm" name="idm46444346174864"></a><p>
        You can move databases or tables from the database directory to
        other locations and replace them with symbolic links to the new
        locations. You might want to do this, for example, to move a
        database to a file system with more free space or increase the
        speed of your system by spreading your tables to different
        disks.
      </p><p>
        For <code class="literal">InnoDB</code> tables, use the <code class="literal">DATA
        DIRECTORY</code> clause of the <a class="link" href="sql-statements.html#create-table" title="13.1.20 CREATE TABLE Statement"><code class="literal">CREATE
        TABLE</code></a> statement instead of symbolic links, as
        explained in <a class="xref" href="innodb-storage-engine.html#innodb-create-table-external" title="15.6.1.2 Creating Tables Externally">Section 15.6.1.2, “Creating Tables Externally”</a>.
        This new feature is a supported, cross-platform technique.
      </p><p>
        The recommended way to do this is to symlink entire database
        directories to a different disk. Symlink
        <code class="literal">MyISAM</code> tables only as a last resort.
      </p><p>
        To determine the location of your data directory, use this
        statement:
</p><pre data-lang="sql" class="programlisting">SHOW VARIABLES LIKE 'datadir';</pre>
<div class="section">

<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="symbolic-links-to-databases"></a>8.12.2.1 Using Symbolic Links for Databases on Unix</h4>
</div>
</div>
</div>
<a class="indexterm" name="idm46444346164784"></a><a class="indexterm" name="idm46444346163296"></a><p>
          On Unix, symlink a database using this procedure:
</p>
<div class="orderedlist">
<ol class="orderedlist" type="1"><li class="listitem"><p>
              Create the database using <a class="link" href="sql-statements.html#create-database" title="13.1.12 CREATE DATABASE Statement"><code class="literal">CREATE
              DATABASE</code></a>:
            </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>CREATE DATABASE mydb1;</code></strong>
</pre><p>
              Using <a class="link" href="sql-statements.html#create-database" title="13.1.12 CREATE DATABASE Statement"><code class="literal">CREATE DATABASE</code></a>
              creates the database in the MySQL data directory and
              permits the server to update the data dictionary with
              information about the database directory.
            </p></li><li class="listitem"><p>
              Stop the server to ensure that no activity occurs in the
              new database while it is being moved.
            </p></li><li class="listitem"><p>
              Move the database directory to some disk where you have
              free space. For example, use <span class="command"><strong>tar</strong></span> or
              <span class="command"><strong>mv</strong></span>. If you use a method that copies
              rather than moves the database directory, remove the
              original database directory after copying it.
            </p></li><li class="listitem"><p>
              Create a soft link in the data directory to the moved
              database directory:
            </p><pre data-lang="terminal" class="programlisting">shell&gt; <strong class="userinput"><code>ln -s <em class="replaceable"><code>/path/to/mydb1</code></em> <em class="replaceable"><code>/path/to/datadir</code></em></code></strong>
</pre><p>
              The command creates a symlink named
              <code class="filename">mydb1</code> in the data directory.
            </p></li><li class="listitem"><p>
              Restart the server.
</p></li></ol>
</div>

</div>

<div class="section">

<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="symbolic-links-to-tables"></a>8.12.2.2 Using Symbolic Links for MyISAM Tables on Unix</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444346144944"></a><a class="indexterm" name="idm46444346143456"></a>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
            Symbolic link support as described here, along with the
            <a class="link" href="server-administration.html#option_mysqld_symbolic-links"><code class="option">--symbolic-links</code></a> option that
            controls it, is deprecated and will be removed in a future
            version of MySQL. In addition, the option is disabled by
            default.
</p>
</div>
<p>
          Symlinks are fully supported only for
          <code class="literal">MyISAM</code> tables. For files used by tables for
          other storage engines, you may get strange problems if you try
          to use symbolic links. For <code class="literal">InnoDB</code> tables,
          use the alternative technique explained in
          <a class="xref" href="innodb-storage-engine.html#innodb-create-table-external" title="15.6.1.2 Creating Tables Externally">Section 15.6.1.2, “Creating Tables Externally”</a> instead.
        </p><p>
          Do not symlink tables on systems that do not have a fully
          operational <code class="literal">realpath()</code> call. (Linux and
          Solaris support <code class="literal">realpath()</code>). To determine
          whether your system supports symbolic links, check the value
          of the <a class="link" href="server-administration.html#sysvar_have_symlink"><code class="literal">have_symlink</code></a> system
          variable using this statement:
        </p><pre data-lang="sql" class="programlisting">SHOW VARIABLES LIKE 'have_symlink';</pre><p>
          The handling of symbolic links for <code class="literal">MyISAM</code>
          tables works as follows:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              In the data directory, you always have the data
              (<code class="filename">.MYD</code>) file and the index
              (<code class="filename">.MYI</code>) file. The data file and index
              file can be moved elsewhere and replaced in the data
              directory by symlinks.
            </p></li><li class="listitem"><p>
              You can symlink the data file and the index file
              independently to different directories.
            </p></li><li class="listitem"><p>
              To instruct a running MySQL server to perform the
              symlinking, use the <code class="literal">DATA DIRECTORY</code> and
              <code class="literal">INDEX DIRECTORY</code> options to
              <a class="link" href="sql-statements.html#create-table" title="13.1.20 CREATE TABLE Statement"><code class="literal">CREATE TABLE</code></a>. See
              <a class="xref" href="sql-statements.html#create-table" title="13.1.20 CREATE TABLE Statement">Section 13.1.20, “CREATE TABLE Statement”</a>. Alternatively, if
              <a class="link" href="programs.html#mysqld" title="4.3.1 mysqld — The MySQL Server"><span class="command"><strong>mysqld</strong></span></a> is not running, symlinking can
              be accomplished manually using <span class="command"><strong>ln -s</strong></span>
              from the command line.
</p>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
                The path used with either or both of the <code class="literal">DATA
                DIRECTORY</code> and <code class="literal">INDEX
                DIRECTORY</code> options may not include the MySQL
                <code class="literal">data</code> directory. (Bug #32167)
</p>
</div>
</li><li class="listitem"><p>
              <a class="link" href="programs.html#myisamchk" title="4.6.4 myisamchk — MyISAM Table-Maintenance Utility"><span class="command"><strong>myisamchk</strong></span></a> does not replace a symlink
              with the data file or index file. It works directly on the
              file to which the symlink points. Any temporary files are
              created in the directory where the data file or index file
              is located. The same is true for the
              <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER TABLE</code></a>,
              <a class="link" href="sql-statements.html#optimize-table" title="13.7.3.4 OPTIMIZE TABLE Statement"><code class="literal">OPTIMIZE TABLE</code></a>, and
              <a class="link" href="sql-statements.html#repair-table" title="13.7.3.5 REPAIR TABLE Statement"><code class="literal">REPAIR TABLE</code></a> statements.
</p></li><li class="listitem">
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
                When you drop a table that is using symlinks,
                <span class="emphasis"><em>both the symlink and the file to which the
                symlink points are dropped</em></span>. This is an
                extremely good reason <span class="emphasis"><em>not</em></span> to run
                <a class="link" href="programs.html#mysqld" title="4.3.1 mysqld — The MySQL Server"><span class="command"><strong>mysqld</strong></span></a> as the <code class="literal">root</code>
                operating system user or permit operating system users
                to have write access to MySQL database directories.
</p>
</div>
</li><li class="listitem"><p>
              If you rename a table with
              <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER TABLE
              ... RENAME</code></a> or <a class="link" href="sql-statements.html#rename-table" title="13.1.36 RENAME TABLE Statement"><code class="literal">RENAME
              TABLE</code></a> and you do not move the table to another
              database, the symlinks in the database directory are
              renamed to the new names and the data file and index file
              are renamed accordingly.
            </p></li><li class="listitem"><p>
              If you use
              <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER TABLE
              ... RENAME</code></a> or <a class="link" href="sql-statements.html#rename-table" title="13.1.36 RENAME TABLE Statement"><code class="literal">RENAME
              TABLE</code></a> to move a table to another database, the
              table is moved to the other database directory. If the
              table name changed, the symlinks in the new database
              directory are renamed to the new names and the data file
              and index file are renamed accordingly.
            </p></li><li class="listitem"><p>
              If you are not using symlinks, start
              <a class="link" href="programs.html#mysqld" title="4.3.1 mysqld — The MySQL Server"><span class="command"><strong>mysqld</strong></span></a> with the
              <a class="link" href="server-administration.html#option_mysqld_symbolic-links"><code class="option">--skip-symbolic-links</code></a>
              option to ensure that no one can use
              <a class="link" href="programs.html#mysqld" title="4.3.1 mysqld — The MySQL Server"><span class="command"><strong>mysqld</strong></span></a> to drop or rename a file outside
              of the data directory.
</p></li></ul>
</div>
<p>
          These table symlink operations are not supported:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER TABLE</code></a> ignores the
              <code class="literal">DATA DIRECTORY</code> and <code class="literal">INDEX
              DIRECTORY</code> table options.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="windows-symbolic-links"></a>8.12.2.3 Using Symbolic Links for Databases on Windows</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444346091040"></a><a class="indexterm" name="idm46444346090000"></a><a class="indexterm" name="idm46444346088512"></a><a class="indexterm" name="idm46444346087424"></a><p>
          On Windows, symbolic links can be used for database
          directories. This enables you to put a database directory at a
          different location (for example, on a different disk) by
          setting up a symbolic link to it. Use of database symlinks on
          Windows is similar to their use on Unix, although the
          procedure for setting up the link differs.
        </p><p>
          Suppose that you want to place the database directory for a
          database named <code class="literal">mydb</code> at
          <code class="filename">D:\data\mydb</code>. To do this, create a
          symbolic link in the MySQL data directory that points to
          <code class="filename">D:\data\mydb</code>. However, before creating
          the symbolic link, make sure that the
          <code class="filename">D:\data\mydb</code> directory exists by creating
          it if necessary. If you already have a database directory
          named <code class="filename">mydb</code> in the data directory, move it
          to <code class="filename">D:\data</code>. Otherwise, the symbolic link
          will be ineffective. To avoid problems, make sure that the
          server is not running when you move the database directory.
        </p><p>
          On Windows, you can create a symlink using the
          <span class="command"><strong>mklink</strong></span> command. This command requires
          administrative privileges.
</p>
<div class="orderedlist">
<ol class="orderedlist" type="1"><li class="listitem"><p>
              Change location into the data directory:
            </p><pre data-lang="terminal" class="programlisting">C:\&gt; <strong class="userinput"><code>cd <em class="replaceable"><code>\path\to\datadir</code></em></code></strong>
</pre></li><li class="listitem"><p>
              In the data directory, create a symlink named
              <code class="filename">mydb</code> that points to the location of
              the database directory:
            </p><pre data-lang="terminal" class="programlisting">C:\&gt; <strong class="userinput"><code>mklink /d mydb D:\data\mydb</code></strong>
</pre></li></ol>
</div>
<p>
          After this, all tables created in the database
          <code class="literal">mydb</code> are created in
          <code class="filename">D:\data\mydb</code>.
</p>
</div>

</div>

<div class="section">

<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="optimizing-memory"></a>8.12.3 Optimizing Memory Use</h3>

</div>

</div>

</div>

<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#memory-use">8.12.3.1 How MySQL Uses Memory</a></span></dt><dt><span class="section"><a href="optimization.html#large-page-support">8.12.3.2 Enabling Large Page Support</a></span></dt></dl>
</div>
<a class="indexterm" name="idm46444346068320"></a>
<div class="section">

<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="memory-use"></a>8.12.3.1 How MySQL Uses Memory</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444346065728"></a><p>
          MySQL allocates buffers and caches to improve performance of
          database operations. The default configuration is designed to
          permit a MySQL server to start on a virtual machine that has
          approximately 512MB of RAM. You can improve MySQL performance
          by increasing the values of certain cache and buffer-related
          system variables. You can also modify the default
          configuration to run MySQL on systems with limited memory.
        </p><p>
          The following list describes some of the ways that MySQL uses
          memory. Where applicable, relevant system variables are
          referenced. Some items are storage engine or feature specific.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
              The <code class="literal">InnoDB</code> buffer pool is a memory area
              that holds cached <code class="literal">InnoDB</code> data for
              tables, indexes, and other auxiliary buffers. For
              efficiency of high-volume read operations, the buffer pool
              is divided into <a class="link" href="glossary.html#glos_page" title="page">pages</a>
              that can potentially hold multiple rows. For efficiency of
              cache management, the buffer pool is implemented as a
              linked list of pages; data that is rarely used is aged out
              of the cache, using a variation of the
              <a class="link" href="glossary.html#glos_lru" title="LRU">LRU</a> algorithm. For more
              information, see <a class="xref" href="innodb-storage-engine.html#innodb-buffer-pool" title="15.5.1 Buffer Pool">Section 15.5.1, “Buffer Pool”</a>.
            </p><p>
              The size of the buffer pool is important for system
              performance:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                  <code class="literal">InnoDB</code> allocates memory for the
                  entire buffer pool at server startup, using
                  <code class="literal">malloc()</code> operations. The
                  <a class="link" href="innodb-storage-engine.html#sysvar_innodb_buffer_pool_size"><code class="literal">innodb_buffer_pool_size</code></a>
                  system variable defines the buffer pool size.
                  Typically, a recommended
                  <a class="link" href="innodb-storage-engine.html#sysvar_innodb_buffer_pool_size"><code class="literal">innodb_buffer_pool_size</code></a>
                  value is 50 to 75 percent of system memory.
                  <a class="link" href="innodb-storage-engine.html#sysvar_innodb_buffer_pool_size"><code class="literal">innodb_buffer_pool_size</code></a>
                  can be configured dynamically, while the server is
                  running. For more information, see
                  <a class="xref" href="innodb-storage-engine.html#innodb-buffer-pool-resize" title="15.8.3.1 Configuring InnoDB Buffer Pool Size">Section 15.8.3.1, “Configuring InnoDB Buffer Pool Size”</a>.
                </p></li><li class="listitem"><p>
                  On systems with a large amount of memory, you can
                  improve concurrency by dividing the buffer pool into
                  multiple
                  <a class="link" href="glossary.html#glos_buffer_pool_instance" title="buffer pool instance">buffer pool
                  instances</a>. The
                  <a class="link" href="innodb-storage-engine.html#sysvar_innodb_buffer_pool_instances"><code class="literal">innodb_buffer_pool_instances</code></a>
                  system variable defines the number of buffer pool
                  instances.
                </p></li><li class="listitem"><p>
                  A buffer pool that is too small may cause excessive
                  churning as pages are flushed from the buffer pool
                  only to be required again a short time later.
                </p></li><li class="listitem"><p>
                  A buffer pool that is too large may cause swapping due
                  to competition for memory.
</p></li></ul>
</div>
</li><li class="listitem"><p>
              The storage engine interface enables the optimizer to
              provide information about the size of the record buffer to
              be used for scans that the optimizer estimates will read
              multiple rows. The buffer size can vary based on the size
              of the estimate. <code class="literal">InnoDB</code> uses this
              variable-size buffering capability to take advantage of
              row prefetching, and to reduce the overhead of latching
              and B-tree navigation.
            </p></li><li class="listitem"><p>
              All threads share the <a class="link" href="storage-engines.html#myisam-storage-engine" title="16.2 The MyISAM Storage Engine"><code class="literal">MyISAM</code></a>
              key buffer. The
              <a class="link" href="server-administration.html#sysvar_key_buffer_size"><code class="literal">key_buffer_size</code></a> system
              variable determines its size.
            </p><p>
              For each <code class="literal">MyISAM</code> table the server opens,
              the index file is opened once; the data file is opened
              once for each concurrently running thread that accesses
              the table. For each concurrent thread, a table structure,
              column structures for each column, and a buffer of size
              <code class="literal">3 * <em class="replaceable"><code>N</code></em></code> are
              allocated (where <em class="replaceable"><code>N</code></em> is the
              maximum row length, not counting
              <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a> columns). A
              <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a> column requires five
              to eight bytes plus the length of the
              <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a> data. The
              <code class="literal">MyISAM</code> storage engine maintains one
              extra row buffer for internal use.
            </p></li><li class="listitem"><p>
              The <a class="link" href="server-administration.html#sysvar_myisam_use_mmap"><code class="literal">myisam_use_mmap</code></a>
              system variable can be set to 1 to enable memory-mapping
              for all <code class="literal">MyISAM</code> tables.
            </p></li><li class="listitem"><p>
              If an internal in-memory temporary table becomes too large
              (as determined using the
              <a class="link" href="server-administration.html#sysvar_tmp_table_size"><code class="literal">tmp_table_size</code></a> and
              <a class="link" href="server-administration.html#sysvar_max_heap_table_size"><code class="literal">max_heap_table_size</code></a>
              system variables), MySQL automatically converts the table
              from in-memory to on-disk format. As of MySQL 8.0.16,
              on-disk temporary tables always use the InnoDB storage
              engine. (Previously, the storage engine employed for this
              purpose was determined by the
              <a class="link" href="server-administration.html#sysvar_internal_tmp_disk_storage_engine"><code class="literal">internal_tmp_disk_storage_engine</code></a>
              system variable, which is no longer supported.) You can
              increase the permissible temporary table size as described
              in <a class="xref" href="optimization.html#internal-temporary-tables" title="8.4.4 Internal Temporary Table Use in MySQL">Section 8.4.4, “Internal Temporary Table Use in MySQL”</a>.
            </p><p>
              For <a class="link" href="storage-engines.html#memory-storage-engine" title="16.3 The MEMORY Storage Engine"><code class="literal">MEMORY</code></a> tables explicitly
              created with <a class="link" href="sql-statements.html#create-table" title="13.1.20 CREATE TABLE Statement"><code class="literal">CREATE TABLE</code></a>,
              only the
              <a class="link" href="server-administration.html#sysvar_max_heap_table_size"><code class="literal">max_heap_table_size</code></a>
              system variable determines how large a table can grow, and
              there is no conversion to on-disk format.
            </p></li><li class="listitem"><p>
              The <a class="link" href="performance-schema.html" title="Chapter 26 MySQL Performance Schema">MySQL Performance
              Schema</a> is a feature for monitoring MySQL server
              execution at a low level. The Performance Schema
              dynamically allocates memory incrementally, scaling its
              memory use to actual server load, instead of allocating
              required memory during server startup. Once memory is
              allocated, it is not freed until the server is restarted.
              For more information, see
              <a class="xref" href="performance-schema.html#performance-schema-memory-model" title="26.17 The Performance Schema Memory-Allocation Model">Section 26.17, “The Performance Schema Memory-Allocation Model”</a>.
            </p></li><li class="listitem"><p>
              Each thread that the server uses to manage client
              connections requires some thread-specific space. The
              following list indicates these and which system variables
              control their size:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                  A stack
                  (<a class="link" href="server-administration.html#sysvar_thread_stack"><code class="literal">thread_stack</code></a>)
                </p></li><li class="listitem"><p>
                  A connection buffer
                  (<a class="link" href="server-administration.html#sysvar_net_buffer_length"><code class="literal">net_buffer_length</code></a>)
                </p></li><li class="listitem"><p>
                  A result buffer
                  (<a class="link" href="server-administration.html#sysvar_net_buffer_length"><code class="literal">net_buffer_length</code></a>)
</p></li></ul>
</div>
<p>
              The connection buffer and result buffer each begin with a
              size equal to
              <a class="link" href="server-administration.html#sysvar_net_buffer_length"><code class="literal">net_buffer_length</code></a> bytes,
              but are dynamically enlarged up to
              <a class="link" href="server-administration.html#sysvar_max_allowed_packet"><code class="literal">max_allowed_packet</code></a> bytes
              as needed. The result buffer shrinks to
              <a class="link" href="server-administration.html#sysvar_net_buffer_length"><code class="literal">net_buffer_length</code></a> bytes
              after each SQL statement. While a statement is running, a
              copy of the current statement string is also allocated.
            </p><p>
              Each connection thread uses memory for computing statement
              digests. The server allocates
              <a class="link" href="server-administration.html#sysvar_max_digest_length"><code class="literal">max_digest_length</code></a> bytes
              per session. See
              <a class="xref" href="performance-schema.html#performance-schema-statement-digests" title="26.10 Performance Schema Statement Digests and Sampling">Section 26.10, “Performance Schema Statement Digests and Sampling”</a>.
            </p></li><li class="listitem"><p>
              All threads share the same base memory.
            </p></li><li class="listitem"><p>
              When a thread is no longer needed, the memory allocated to
              it is released and returned to the system unless the
              thread goes back into the thread cache. In that case, the
              memory remains allocated.
            </p></li><li class="listitem"><p>
              Each request that performs a sequential scan of a table
              allocates a <span class="firstterm">read
              buffer</span>. The
              <a class="link" href="server-administration.html#sysvar_read_buffer_size"><code class="literal">read_buffer_size</code></a> system
              variable determines the buffer size.
            </p></li><li class="listitem"><p>
              When reading rows in an arbitrary sequence (for example,
              following a sort), a
              <span class="firstterm">random-read buffer</span>
              may be allocated to avoid disk seeks. The
              <a class="link" href="server-administration.html#sysvar_read_rnd_buffer_size"><code class="literal">read_rnd_buffer_size</code></a>
              system variable determines the buffer size.
            </p></li><li class="listitem"><p>
              All joins are executed in a single pass, and most joins
              can be done without even using a temporary table. Most
              temporary tables are memory-based hash tables. Temporary
              tables with a large row length (calculated as the sum of
              all column lengths) or that contain
              <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a> columns are stored on
              disk.
            </p></li><li class="listitem"><p>
              Most requests that perform a sort allocate a sort buffer
              and zero to two temporary files depending on the result
              set size. See <a class="xref" href="error-handling.html#temporary-files" title="B.4.3.5 Where MySQL Stores Temporary Files">Section B.4.3.5, “Where MySQL Stores Temporary Files”</a>.
            </p></li><li class="listitem"><p>
              Almost all parsing and calculating is done in thread-local
              and reusable memory pools. No memory overhead is needed
              for small items, thus avoiding the normal slow memory
              allocation and freeing. Memory is allocated only for
              unexpectedly large strings.
            </p></li><li class="listitem"><p>
              For each table having <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a>
              columns, a buffer is enlarged dynamically to read in
              larger <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a> values. If you
              scan a table, the buffer grows as large as the largest
              <a class="link" href="data-types.html#blob" title="11.3.4 The BLOB and TEXT Types"><code class="literal">BLOB</code></a> value.
            </p></li><li class="listitem"><p>
              MySQL requires memory and descriptors for the table cache.
              Handler structures for all in-use tables are saved in the
              table cache and managed as <span class="quote">“<span class="quote">First In, First
              Out</span>”</span> (FIFO). The
              <a class="link" href="server-administration.html#sysvar_table_open_cache"><code class="literal">table_open_cache</code></a> system
              variable defines the initial table cache size; see
              <a class="xref" href="optimization.html#table-cache" title="8.4.3.1 How MySQL Opens and Closes Tables">Section 8.4.3.1, “How MySQL Opens and Closes Tables”</a>.
            </p><p>
              MySQL also requires memory for the table definition cache.
              The
              <a class="link" href="server-administration.html#sysvar_table_definition_cache"><code class="literal">table_definition_cache</code></a>
              system variable defines the number of table definitions
              that can be stored in the table definition cache. If you
              use a large number of tables, you can create a large table
              definition cache to speed up the opening of tables. The
              table definition cache takes less space and does not use
              file descriptors, unlike the table cache.
            </p></li><li class="listitem"><p>
              A <a class="link" href="sql-statements.html#flush-tables"><code class="literal">FLUSH TABLES</code></a> statement or
              <a class="link" href="programs.html#mysqladmin" title="4.5.2 mysqladmin — Client for Administering a MySQL Server"><span class="command"><strong>mysqladmin flush-tables</strong></span></a> command closes
              all tables that are not in use at once and marks all
              in-use tables to be closed when the currently executing
              thread finishes. This effectively frees most in-use
              memory. <a class="link" href="sql-statements.html#flush-tables"><code class="literal">FLUSH TABLES</code></a> does
              not return until all tables have been closed.
            </p></li><li class="listitem"><p>
              The server caches information in memory as a result of
              <a class="link" href="sql-statements.html#grant" title="13.7.1.6 GRANT Statement"><code class="literal">GRANT</code></a>,
              <a class="link" href="sql-statements.html#create-user" title="13.7.1.3 CREATE USER Statement"><code class="literal">CREATE USER</code></a>,
              <a class="link" href="sql-statements.html#create-server" title="13.1.18 CREATE SERVER Statement"><code class="literal">CREATE SERVER</code></a>, and
              <a class="link" href="sql-statements.html#install-plugin" title="13.7.4.4 INSTALL PLUGIN Statement"><code class="literal">INSTALL PLUGIN</code></a> statements.
              This memory is not released by the corresponding
              <a class="link" href="sql-statements.html#revoke" title="13.7.1.8 REVOKE Statement"><code class="literal">REVOKE</code></a>,
              <a class="link" href="sql-statements.html#drop-user" title="13.7.1.5 DROP USER Statement"><code class="literal">DROP USER</code></a>,
              <a class="link" href="sql-statements.html#drop-server" title="13.1.30 DROP SERVER Statement"><code class="literal">DROP SERVER</code></a>, and
              <a class="link" href="sql-statements.html#uninstall-plugin" title="13.7.4.6 UNINSTALL PLUGIN Statement"><code class="literal">UNINSTALL PLUGIN</code></a>
              statements, so for a server that executes many instances
              of the statements that cause caching, there will be an
              increase in cached memory use unless it is freed with
              <a class="link" href="sql-statements.html#flush-privileges"><code class="literal">FLUSH PRIVILEGES</code></a>.
            </p></li><li class="listitem"><p>
              In a replication topology, the following settings affect
              memory usage, and can be adjusted as required:

</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                    The
                    <a class="link" href="server-administration.html#sysvar_max_allowed_packet"><code class="literal">max_allowed_packet</code></a>
                    system variable on a replication master limits the
                    maximum message size that the master sends to its
                    slaves for processing. This setting defaults to 64M.
                  </p></li><li class="listitem"><p>
                    The
                    <a class="link" href="replication.html#sysvar_slave_pending_jobs_size_max"><code class="literal">slave_pending_jobs_size_max</code></a>
                    system variable on a multi-threaded slave sets the
                    maximum amount of memory that is made available for
                    holding messages awaiting processing. This setting
                    defaults to 128M. The memory is only allocated when
                    needed, but it might be used if your replication
                    topology handles large transactions sometimes. It is
                    a soft limit, and larger transactions can be
                    processed.
                  </p></li><li class="listitem"><p>
                    The <a class="link" href="replication.html#sysvar_rpl_read_size"><code class="literal">rpl_read_size</code></a>
                    system variable on a replication master or slave
                    controls the minimum amount of data in bytes that is
                    read from the binary log files and relay log files.
                    The default is 8192 bytes. A buffer the size of this
                    value is allocated for each thread that reads from
                    the binary log and relay log files, including dump
                    threads on masters and coordinator threads on
                    slaves.
                  </p></li><li class="listitem"><p>
                    The
                    <a class="link" href="replication.html#sysvar_binlog_transaction_dependency_history_size"><code class="literal">binlog_transaction_dependency_history_size</code></a>
                    system variable limits the number of row hashes held
                    as an in-memory history.
                  </p></li><li class="listitem"><p>
                    The
                    <a class="link" href="replication.html#sysvar_max_binlog_cache_size"><code class="literal">max_binlog_cache_size</code></a>
                    system variable specifies the upper limit of memory
                    usage by an individual transaction.
                  </p></li><li class="listitem"><p>
                    The
                    <a class="link" href="replication.html#sysvar_max_binlog_stmt_cache_size"><code class="literal">max_binlog_stmt_cache_size</code></a>
                    system variable specifies the upper limit of memory
                    usage by the statement cache.
</p></li></ul>
</div>
<p>
</p></li></ul>
</div>
<p>
          <span class="command"><strong>ps</strong></span> and other system status programs may
          report that <a class="link" href="programs.html#mysqld" title="4.3.1 mysqld — The MySQL Server"><span class="command"><strong>mysqld</strong></span></a> uses a lot of memory.
          This may be caused by thread stacks on different memory
          addresses. For example, the Solaris version of
          <span class="command"><strong>ps</strong></span> counts the unused memory between stacks
          as used memory. To verify this, check available swap with
          <code class="literal">swap -s</code>. We test <a class="link" href="programs.html#mysqld" title="4.3.1 mysqld — The MySQL Server"><span class="command"><strong>mysqld</strong></span></a>
          with several memory-leakage detectors (both commercial and
          Open Source), so there should be no memory leaks.
</p>
<div class="simplesect">

<div class="titlepage">
<div>

<div class="simple">
<h5 class="title"><a name="monitor-mysql-memory-use"></a>Monitoring MySQL Memory Usage</h5>
</div>
</div>
</div>
<a class="indexterm" name="idm46444345936848"></a><a class="indexterm" name="idm46444345935360"></a><p>
            The following example demonstrates how to use
            <a class="link" href="performance-schema.html" title="Chapter 26 MySQL Performance Schema">Performance Schema</a>
            and <a class="link" href="sys-schema.html" title="Chapter 27 MySQL sys Schema">sys schema</a> to monitor
            MySQL memory usage.
          </p><p>
            Most Performance Schema memory instrumentation is disabled
            by default. Instruments can be enabled by updating the
            <code class="literal">ENABLED</code> column of the Performance Schema
            <a class="link" href="performance-schema.html#setup-instruments-table" title="26.12.2.3 The setup_instruments Table"><code class="literal">setup_instruments</code></a> table. Memory
            instruments have names in the form of
            <code class="literal">memory/<em class="replaceable"><code>code_area</code></em>/<em class="replaceable"><code>instrument_name</code></em></code>,
            where <em class="replaceable"><code>code_area</code></em> is a value such
            as <code class="literal">sql</code> or <code class="literal">innodb</code>, and
            <em class="replaceable"><code>instrument_name</code></em> is the instrument
            detail.
</p>
<div class="orderedlist">
<ol class="orderedlist" type="1"><li class="listitem"><p>
                To view available MySQL memory instruments, query the
                Performance Schema
                <a class="link" href="performance-schema.html#setup-instruments-table" title="26.12.2.3 The setup_instruments Table"><code class="literal">setup_instruments</code></a> table.
                The following query returns hundreds of memory
                instruments for all code areas.
              </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT * FROM performance_schema.setup_instruments</code></strong>
       <strong class="userinput"><code>WHERE NAME LIKE '%memory%';</code></strong></pre><p>
                You can narrow results by specifying a code area. For
                example, you can limit results to
                <code class="literal">InnoDB</code> memory instruments by
                specifying <code class="literal">innodb</code> as the code area.
              </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT * FROM performance_schema.setup_instruments</code></strong>
       <strong class="userinput"><code>WHERE NAME LIKE '%memory/innodb%';</code></strong>
+-------------------------------------------+---------+-------+
| NAME                                      | ENABLED | TIMED |
+-------------------------------------------+---------+-------+
| memory/innodb/adaptive hash index         | NO      | NO    |
| memory/innodb/buf_buf_pool                | NO      | NO    |
| memory/innodb/dict_stats_bg_recalc_pool_t | NO      | NO    |
| memory/innodb/dict_stats_index_map_t      | NO      | NO    |
| memory/innodb/dict_stats_n_diff_on_level  | NO      | NO    |
| memory/innodb/other                       | NO      | NO    |
| memory/innodb/row_log_buf                 | NO      | NO    |
| memory/innodb/row_merge_sort              | NO      | NO    |
| memory/innodb/std                         | NO      | NO    |
| memory/innodb/trx_sys_t::rw_trx_ids       | NO      | NO    |
...
</pre><p>
                Depending on your MySQL installation, code areas may
                include <code class="literal">performance_schema</code>,
                <code class="literal">sql</code>, <code class="literal">client</code>,
                <code class="literal">innodb</code>, <code class="literal">myisam</code>,
                <code class="literal">csv</code>, <code class="literal">memory</code>,
                <code class="literal">blackhole</code>,
                <code class="literal">archive</code>,
                <code class="literal">partition</code>, and others.
              </p></li><li class="listitem"><p>
                To enable memory instruments, add a
                <code class="literal">performance-schema-instrument</code> rule to
                your MySQL configuration file. For example, to enable
                all memory instruments, add this rule to your
                configuration file and restart the server:
</p><pre data-lang="ini" class="programlisting">performance-schema-instrument='memory/%=COUNTED'</pre>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
                  Enabling memory instruments at startup ensures that
                  memory allocations that occur at startup are counted.
</p>
</div>
<p>
                After restarting the server, the
                <code class="literal">ENABLED</code> column of the Performance
                Schema <a class="link" href="performance-schema.html#setup-instruments-table" title="26.12.2.3 The setup_instruments Table"><code class="literal">setup_instruments</code></a>
                table should report <code class="literal">YES</code> for memory
                instruments that you enabled. The
                <code class="literal">TIMED</code> column in the
                <a class="link" href="performance-schema.html#setup-instruments-table" title="26.12.2.3 The setup_instruments Table"><code class="literal">setup_instruments</code></a> table is
                ignored for memory instruments because memory operations
                are not timed.
              </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT * FROM performance_schema.setup_instruments</code></strong>
       <strong class="userinput"><code>WHERE NAME LIKE '%memory/innodb%';</code></strong>
+-------------------------------------------+---------+-------+
| NAME                                      | ENABLED | TIMED |
+-------------------------------------------+---------+-------+
| memory/innodb/adaptive hash index         | NO      | NO    |
| memory/innodb/buf_buf_pool                | NO      | NO    |
| memory/innodb/dict_stats_bg_recalc_pool_t | NO      | NO    |
| memory/innodb/dict_stats_index_map_t      | NO      | NO    |
| memory/innodb/dict_stats_n_diff_on_level  | NO      | NO    |
| memory/innodb/other                       | NO      | NO    |
| memory/innodb/row_log_buf                 | NO      | NO    |
| memory/innodb/row_merge_sort              | NO      | NO    |
| memory/innodb/std                         | NO      | NO    |
| memory/innodb/trx_sys_t::rw_trx_ids       | NO      | NO    |
...
</pre></li><li class="listitem"><p>
                Query memory instrument data. In this example, memory
                instrument data is queried in the Performance Schema
                <a class="link" href="performance-schema.html#memory-summary-tables" title="26.12.18.10 Memory Summary Tables"><code class="literal">memory_summary_global_by_event_name</code></a>
                table, which summarizes data by
                <code class="literal">EVENT_NAME</code>. The
                <code class="literal">EVENT_NAME</code> is the name of the
                instrument.
              </p><p>
                The following query returns memory data for the
                <code class="literal">InnoDB</code> buffer pool. For column
                descriptions, see
                <a class="xref" href="performance-schema.html#memory-summary-tables" title="26.12.18.10 Memory Summary Tables">Section 26.12.18.10, “Memory Summary Tables”</a>.
              </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT * FROM performance_schema.memory_summary_global_by_event_name</code></strong>
       <strong class="userinput"><code>WHERE EVENT_NAME LIKE 'memory/innodb/buf_buf_pool'\G</code></strong>
                  EVENT_NAME: memory/innodb/buf_buf_pool
                 COUNT_ALLOC: 1
                  COUNT_FREE: 0
   SUM_NUMBER_OF_BYTES_ALLOC: 137428992
    SUM_NUMBER_OF_BYTES_FREE: 0
              LOW_COUNT_USED: 0
          CURRENT_COUNT_USED: 1
             HIGH_COUNT_USED: 1
    LOW_NUMBER_OF_BYTES_USED: 0
CURRENT_NUMBER_OF_BYTES_USED: 137428992
   HIGH_NUMBER_OF_BYTES_USED: 137428992
</pre><p>
                The same underlying data can be queried using the
                <a class="link" href="sys-schema.html" title="Chapter 27 MySQL sys Schema"><code class="literal">sys</code></a> schema
                <a class="link" href="sys-schema.html#sys-memory-global-by-current-bytes" title="27.4.3.19 The memory_global_by_current_bytes and x$memory_global_by_current_bytes Views"><code class="literal">memory_global_by_current_bytes</code></a>
                table, which shows current memory usage within the
                server globally, broken down by allocation type.
              </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT * FROM sys.memory_global_by_current_bytes</code></strong>
       <strong class="userinput"><code>WHERE event_name LIKE 'memory/innodb/buf_buf_pool'\G</code></strong>
*************************** 1. row ***************************
       event_name: memory/innodb/buf_buf_pool
    current_count: 1
    current_alloc: 131.06 MiB
current_avg_alloc: 131.06 MiB
       high_count: 1
       high_alloc: 131.06 MiB
   high_avg_alloc: 131.06 MiB
</pre><p>
                This <a class="link" href="sys-schema.html" title="Chapter 27 MySQL sys Schema"><code class="literal">sys</code></a> schema query
                aggregates currently allocated memory
                (<code class="literal">current_alloc</code>) by code area:
              </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT SUBSTRING_INDEX(event_name,'/',2) AS</code></strong>
       <strong class="userinput"><code>code_area, FORMAT_BYTES(SUM(current_alloc))</code></strong>
       <strong class="userinput"><code>AS current_alloc</code></strong>
       <strong class="userinput"><code>FROM sys.x$memory_global_by_current_bytes</code></strong>
       <strong class="userinput"><code>GROUP BY SUBSTRING_INDEX(event_name,'/',2)</code></strong>
       <strong class="userinput"><code>ORDER BY SUM(current_alloc) DESC;</code></strong>
+---------------------------+---------------+
| code_area                 | current_alloc |
+---------------------------+---------------+
| memory/innodb             | 843.24 MiB    |
| memory/performance_schema | 81.29 MiB     |
| memory/mysys              | 8.20 MiB      |
| memory/sql                | 2.47 MiB      |
| memory/memory             | 174.01 KiB    |
| memory/myisam             | 46.53 KiB     |
| memory/blackhole          | 512 bytes     |
| memory/federated          | 512 bytes     |
| memory/csv                | 512 bytes     |
| memory/vio                | 496 bytes     |
+---------------------------+---------------+
</pre>
<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">

<div class="admon-title">
Note
</div>
<p>
                  Prior to MySQL 8.0.16,
                  <code class="literal">sys.format_bytes()</code> was used for
                  <a class="link" href="functions.html#function_format-bytes"><code class="literal">FORMAT_BYTES()</code></a>.
</p>
</div>
<p>
                For more information about
                <a class="link" href="sys-schema.html" title="Chapter 27 MySQL sys Schema"><code class="literal">sys</code></a> schema, see
                <a class="xref" href="sys-schema.html" title="Chapter 27 MySQL sys Schema">Chapter 27, <i>MySQL sys Schema</i></a>.
</p></li></ol>
</div>

</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h4 class="title"><a name="large-page-support"></a>8.12.3.2 Enabling Large Page Support</h4>

</div>

</div>

</div>
<a class="indexterm" name="idm46444345867552"></a><p>
          Some hardware/operating system architectures support memory
          pages greater than the default (usually 4KB). The actual
          implementation of this support depends on the underlying
          hardware and operating system. Applications that perform a lot
          of memory accesses may obtain performance improvements by
          using large pages due to reduced Translation Lookaside Buffer
          (TLB) misses.
        </p><p>
          In MySQL, large pages can be used by InnoDB, to allocate
          memory for its buffer pool and additional memory pool.
        </p><p>
          Standard use of large pages in MySQL attempts to use the
          largest size supported, up to 4MB. Under Solaris, a
          <span class="quote">“<span class="quote">super large pages</span>”</span> feature enables uses of pages
          up to 256MB. This feature is available for recent SPARC
          platforms. It can be enabled or disabled by using the
          <a class="link" href="server-administration.html#option_mysqld_super-large-pages"><code class="option">--super-large-pages</code></a> or
          <a class="link" href="server-administration.html#option_mysqld_super-large-pages"><code class="option">--skip-super-large-pages</code></a>
          option.
        </p><p>
          MySQL also supports the Linux implementation of large page
          support (which is called HugeTLB in Linux).
        </p><p>
          Before large pages can be used on Linux, the kernel must be
          enabled to support them and it is necessary to configure the
          HugeTLB memory pool. For reference, the HugeTBL API is
          documented in the
          <code class="filename">Documentation/vm/hugetlbpage.txt</code> file of
          your Linux sources.
        </p><p>
          The kernel for some recent systems such as Red Hat Enterprise
          Linux appear to have the large pages feature enabled by
          default. To check whether this is true for your kernel, use
          the following command and look for output lines containing
          <span class="quote">“<span class="quote">huge</span>”</span>:
        </p><pre data-lang="terminal" class="programlisting">shell&gt; <strong class="userinput"><code>cat /proc/meminfo | grep -i huge</code></strong>
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       4096 kB
</pre><p>
          The nonempty command output indicates that large page support
          is present, but the zero values indicate that no pages are
          configured for use.
        </p><p>
          If your kernel needs to be reconfigured to support large
          pages, consult the <code class="filename">hugetlbpage.txt</code> file
          for instructions.
        </p><p>
          Assuming that your Linux kernel has large page support
          enabled, configure it for use by MySQL using the following
          commands. Normally, you put these in an
          <code class="filename">rc</code> file or equivalent startup file that
          is executed during the system boot sequence, so that the
          commands execute each time the system starts. The commands
          should execute early in the boot sequence, before the MySQL
          server starts. Be sure to change the allocation numbers and
          the group number as appropriate for your system.
        </p><pre data-lang="terminal" class="programlisting"># Set the number of pages to be used.
# Each page is normally 2MB, so a value of 20 = 40MB.
# This command actually allocates memory, so this much
# memory must be available.
echo 20 &gt; /proc/sys/vm/nr_hugepages

# Set the group number that is permitted to access this
# memory (102 in this case). The mysql user must be a
# member of this group.
echo 102 &gt; /proc/sys/vm/hugetlb_shm_group

# Increase the amount of shmem permitted per segment
# (12G in this case).
echo 1560281088 &gt; /proc/sys/kernel/shmmax

# Increase total amount of shared memory.  The value
# is the number of pages. At 4KB/page, 4194304 = 16GB.
echo 4194304 &gt; /proc/sys/kernel/shmall</pre><p>
          For MySQL usage, you normally want the value of
          <code class="literal">shmmax</code> to be close to the value of
          <code class="literal">shmall</code>.
        </p><p>
          To verify the large page configuration, check
          <code class="filename">/proc/meminfo</code> again as described
          previously. Now you should see some nonzero values:
        </p><pre data-lang="terminal" class="programlisting">shell&gt; <strong class="userinput"><code>cat /proc/meminfo | grep -i huge</code></strong>
HugePages_Total:      20
HugePages_Free:       20
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       4096 kB
</pre><p>
          The final step to make use of the
          <code class="literal">hugetlb_shm_group</code> is to give the
          <code class="literal">mysql</code> user an <span class="quote">“<span class="quote">unlimited</span>”</span>
          value for the memlock limit. This can be done either by
          editing <code class="filename">/etc/security/limits.conf</code> or by
          adding the following command to your
          <a class="link" href="programs.html#mysqld-safe" title="4.3.2 mysqld_safe — MySQL Server Startup Script"><span class="command"><strong>mysqld_safe</strong></span></a> script:
        </p><pre data-lang="terminal" class="programlisting">ulimit -l unlimited</pre><p>
          Adding the <span class="command"><strong>ulimit</strong></span> command to
          <a class="link" href="programs.html#mysqld-safe" title="4.3.2 mysqld_safe — MySQL Server Startup Script"><span class="command"><strong>mysqld_safe</strong></span></a> causes the
          <code class="literal">root</code> user to set the memlock limit to
          <code class="literal">unlimited</code> before switching to the
          <code class="literal">mysql</code> user. (This assumes that
          <a class="link" href="programs.html#mysqld-safe" title="4.3.2 mysqld_safe — MySQL Server Startup Script"><span class="command"><strong>mysqld_safe</strong></span></a> is started by
          <code class="literal">root</code>.)
        </p><p>
          Large page support in MySQL is disabled by default. To enable
          it, start the server with the
          <a class="link" href="server-administration.html#option_mysqld_large-pages"><code class="option">--large-pages</code></a> option. For
          example, you can use the following lines in the server
          <code class="filename">my.cnf</code> file:
        </p><pre data-lang="ini" class="programlisting">[mysqld]
large-pages</pre><p>
          With this option, <code class="literal">InnoDB</code> uses large pages
          automatically for its buffer pool and additional memory pool.
          If <code class="literal">InnoDB</code> cannot do this, it falls back to
          use of traditional memory and writes a warning to the error
          log: <span class="errortext">Warning: Using conventional memory
          pool</span>
        </p><p>
          To verify that large pages are being used, check
          <code class="literal">/proc/meminfo</code> again:
        </p><pre data-lang="terminal" class="programlisting">shell&gt; <strong class="userinput"><code>cat /proc/meminfo | grep -i huge</code></strong>
HugePages_Total:      20
HugePages_Free:       20
HugePages_Rsvd:        2
HugePages_Surp:        0
Hugepagesize:       4096 kB
</pre>
</div>

</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a name="optimize-benchmarking"></a>8.13 Measuring Performance (Benchmarking)</h2>

</div>

</div>

</div>
<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#select-benchmarking">8.13.1 Measuring the Speed of Expressions and Functions</a></span></dt><dt><span class="section"><a href="optimization.html#custom-benchmarks">8.13.2 Using Your Own Benchmarks</a></span></dt><dt><span class="section"><a href="optimization.html#monitoring-performance-schema">8.13.3 Measuring Performance with performance_schema</a></span></dt></dl>
</div>
<a class="indexterm" name="idm46444345824928"></a><p>
      To measure performance, consider the following factors:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
          Whether you are measuring the speed of a single operation on a
          quiet system, or how a set of operations (a
          <span class="quote">“<span class="quote">workload</span>”</span>) works over a period of time. With
          simple tests, you usually test how changing one aspect (a
          configuration setting, the set of indexes on a table, the SQL
          clauses in a query) affects performance. Benchmarks are
          typically long-running and elaborate performance tests, where
          the results could dictate high-level choices such as hardware
          and storage configuration, or how soon to upgrade to a new
          MySQL version.
        </p></li><li class="listitem"><p>
          For benchmarking, sometimes you must simulate a heavy database
          workload to get an accurate picture.
        </p></li><li class="listitem"><p>
          Performance can vary depending on so many different factors
          that a difference of a few percentage points might not be a
          decisive victory. The results might shift the opposite way
          when you test in a different environment.
        </p></li><li class="listitem"><p>
          Certain MySQL features help or do not help performance
          depending on the workload. For completeness, always test
          performance with those features turned on and turned off. The
          most important feature to try with each workload is the
          <a class="link" href="innodb-storage-engine.html#innodb-adaptive-hash" title="15.5.3 Adaptive Hash Index">adaptive hash
          index</a> for <code class="literal">InnoDB</code> tables.
</p></li></ul>
</div>
<p>
      This section progresses from simple and direct measurement
      techniques that a single developer can do, to more complicated
      ones that require additional expertise to perform and interpret
      the results.
</p>
<div class="section">

<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="select-benchmarking"></a>8.13.1 Measuring the Speed of Expressions and Functions</h3>
</div>
</div>
</div>
<p>
        To measure the speed of a specific MySQL expression or function,
        invoke the <a class="link" href="functions.html#function_benchmark"><code class="literal">BENCHMARK()</code></a> function
        using the <a class="link" href="programs.html#mysql" title="4.5.1 mysql — The MySQL Command-Line Client"><span class="command"><strong>mysql</strong></span></a> client program. Its syntax is
        <a class="link" href="functions.html#function_benchmark"><code class="literal">BENCHMARK(<em class="replaceable"><code>loop_count</code></em>,<em class="replaceable"><code>expr</code></em>)</code></a>.
        The return value is always zero, but <a class="link" href="programs.html#mysql" title="4.5.1 mysql — The MySQL Command-Line Client"><span class="command"><strong>mysql</strong></span></a>
        prints a line displaying approximately how long the statement
        took to execute. For example:
      </p><pre data-lang="sql" class="programlisting">mysql&gt; <strong class="userinput"><code>SELECT BENCHMARK(1000000,1+1);</code></strong>
+------------------------+
| BENCHMARK(1000000,1+1) |
+------------------------+
|                      0 |
+------------------------+
1 row in set (0.32 sec)
</pre><p>
        This result was obtained on a Pentium II 400MHz system. It shows
        that MySQL can execute 1,000,000 simple addition expressions in
        0.32 seconds on that system.
      </p><p>
        The built-in MySQL functions are typically highly optimized, but
        there may be some exceptions.
        <a class="link" href="functions.html#function_benchmark"><code class="literal">BENCHMARK()</code></a> is an excellent tool
        for finding out if some function is a problem for your queries.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="custom-benchmarks"></a>8.13.2 Using Your Own Benchmarks</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444345802304"></a><a class="indexterm" name="idm46444345801264"></a><p>
        Benchmark your application and database to find out where the
        bottlenecks are. After fixing one bottleneck (or by replacing it
        with a <span class="quote">“<span class="quote">dummy</span>”</span> module), you can proceed to identify
        the next bottleneck. Even if the overall performance for your
        application currently is acceptable, you should at least make a
        plan for each bottleneck and decide how to solve it if someday
        you really need the extra performance.
      </p><p>
        A free benchmark suite is the Open Source Database Benchmark,
        available at <a class="ulink" href="http://osdb.sourceforge.net/" target="_top">http://osdb.sourceforge.net/</a>.
      </p><p>
        It is very common for a problem to occur only when the system is
        very heavily loaded. We have had many customers who contact us
        when they have a (tested) system in production and have
        encountered load problems. In most cases, performance problems
        turn out to be due to issues of basic database design (for
        example, table scans are not good under high load) or problems
        with the operating system or libraries. Most of the time, these
        problems would be much easier to fix if the systems were not
        already in production.
      </p><p>
        To avoid problems like this, benchmark your whole application
        under the worst possible load:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            The <a class="link" href="programs.html#mysqlslap" title="4.5.8 mysqlslap — Load Emulation Client"><span class="command"><strong>mysqlslap</strong></span></a> program can be helpful for
            simulating a high load produced by multiple clients issuing
            queries simultaneously. See <a class="xref" href="programs.html#mysqlslap" title="4.5.8 mysqlslap — Load Emulation Client">Section 4.5.8, “<span class="command"><strong>mysqlslap</strong></span> — Load Emulation Client”</a>.
          </p></li><li class="listitem"><p>
            You can also try benchmarking packages such as SysBench and
            DBT2, available at
            <a class="ulink" href="https://launchpad.net/sysbench" target="_top">https://launchpad.net/sysbench</a>, and
            <a class="ulink" href="http://osdldbt.sourceforge.net/#dbt2" target="_top">http://osdldbt.sourceforge.net/#dbt2</a>.
</p></li></ul>
</div>
<p>
        These programs or packages can bring a system to its knees, so
        be sure to use them only on your development systems.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="monitoring-performance-schema"></a>8.13.3 Measuring Performance with performance_schema</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444345788704"></a><p>
        You can query the tables in the
        <code class="literal">performance_schema</code> database to see real-time
        information about the performance characteristics of your server
        and the applications it is running. See
        <a class="xref" href="performance-schema.html" title="Chapter 26 MySQL Performance Schema">Chapter 26, <i>MySQL Performance Schema</i></a> for details.
</p>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a name="thread-information"></a>8.14 Examining Thread Information</h2>

</div>

</div>

</div>
<div class="toc">
<dl class="toc"><dt><span class="section"><a href="optimization.html#thread-commands">8.14.1 Thread Command Values</a></span></dt><dt><span class="section"><a href="optimization.html#general-thread-states">8.14.2 General Thread States</a></span></dt><dt><span class="section"><a href="optimization.html#master-thread-states">8.14.3 Replication Master Thread States</a></span></dt><dt><span class="section"><a href="optimization.html#slave-io-thread-states">8.14.4 Replication Slave I/O Thread States</a></span></dt><dt><span class="section"><a href="optimization.html#slave-sql-thread-states">8.14.5 Replication Slave SQL Thread States</a></span></dt><dt><span class="section"><a href="optimization.html#slave-connection-thread-states">8.14.6 Replication Slave Connection Thread States</a></span></dt><dt><span class="section"><a href="optimization.html#mysql-cluster-thread-states">8.14.7 NDB Cluster Thread States</a></span></dt><dt><span class="section"><a href="optimization.html#event-scheduler-thread-states">8.14.8 Event Scheduler Thread States</a></span></dt></dl>
</div>
<a class="indexterm" name="idm46444345783872"></a><a class="indexterm" name="idm46444345782416"></a><a class="indexterm" name="idm46444345780928"></a><p>
      When you are attempting to ascertain what your MySQL server is
      doing, it can be helpful to examine the process list, which is the
      set of threads currently executing within the server. Process list
      information is available from these sources:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
          The <code class="literal">SHOW [FULL] PROCESSLIST</code> statement:
          <a class="xref" href="sql-statements.html#show-processlist" title="13.7.7.29 SHOW PROCESSLIST Statement">Section 13.7.7.29, “SHOW PROCESSLIST Statement”</a>
        </p></li><li class="listitem"><p>
          The <a class="link" href="sql-statements.html#show-profile" title="13.7.7.30 SHOW PROFILE Statement"><code class="literal">SHOW PROFILE</code></a> statement:
          <a class="xref" href="sql-statements.html#show-profiles" title="13.7.7.31 SHOW PROFILES Statement">Section 13.7.7.31, “SHOW PROFILES Statement”</a>
        </p></li><li class="listitem"><p>
          The <code class="literal">INFORMATION_SCHEMA</code>
          <a class="link" href="information-schema.html#processlist-table" title="25.22 The INFORMATION_SCHEMA PROCESSLIST Table"><code class="literal">PROCESSLIST</code></a> table:
          <a class="xref" href="information-schema.html#processlist-table" title="25.22 The INFORMATION_SCHEMA PROCESSLIST Table">Section 25.22, “The INFORMATION_SCHEMA PROCESSLIST Table”</a>
        </p></li><li class="listitem"><p>
          The <a class="link" href="programs.html#mysqladmin" title="4.5.2 mysqladmin — Client for Administering a MySQL Server"><span class="command"><strong>mysqladmin processlist</strong></span></a> command:
          <a class="xref" href="programs.html#mysqladmin" title="4.5.2 mysqladmin — Client for Administering a MySQL Server">Section 4.5.2, “<span class="command"><strong>mysqladmin</strong></span> — Client for Administering a MySQL Server”</a>
        </p></li><li class="listitem"><p>
          The Performance Schema <a class="link" href="performance-schema.html#threads-table" title="26.12.19.5 The threads Table"><code class="literal">threads</code></a>
          table, stage tables, and lock tables:
          <a class="xref" href="performance-schema.html#performance-schema-miscellaneous-tables" title="26.12.19 Performance Schema Miscellaneous Tables">Section 26.12.19, “Performance Schema Miscellaneous Tables”</a>,
          <a class="xref" href="performance-schema.html#performance-schema-stage-tables" title="26.12.5 Performance Schema Stage Event Tables">Section 26.12.5, “Performance Schema Stage Event Tables”</a>,
          <a class="xref" href="performance-schema.html#performance-schema-lock-tables" title="26.12.13 Performance Schema Lock Tables">Section 26.12.13, “Performance Schema Lock Tables”</a>.
</p></li></ul>
</div>
<p>
      Access to <a class="link" href="performance-schema.html#threads-table" title="26.12.19.5 The threads Table"><code class="literal">threads</code></a> does not require a
      mutex and has minimal impact on server performance.
      <a class="link" href="information-schema.html#processlist-table" title="25.22 The INFORMATION_SCHEMA PROCESSLIST Table"><code class="literal">INFORMATION_SCHEMA.PROCESSLIST</code></a> and
      <a class="link" href="sql-statements.html#show-processlist" title="13.7.7.29 SHOW PROCESSLIST Statement"><code class="literal">SHOW PROCESSLIST</code></a> have negative
      performance consequences because they require a mutex.
      <a class="link" href="performance-schema.html#threads-table" title="26.12.19.5 The threads Table"><code class="literal">threads</code></a> also shows information about
      background threads, which
      <a class="link" href="information-schema.html#processlist-table" title="25.22 The INFORMATION_SCHEMA PROCESSLIST Table"><code class="literal">INFORMATION_SCHEMA.PROCESSLIST</code></a> and
      <a class="link" href="sql-statements.html#show-processlist" title="13.7.7.29 SHOW PROCESSLIST Statement"><code class="literal">SHOW PROCESSLIST</code></a> do not. This means
      that <a class="link" href="performance-schema.html#threads-table" title="26.12.19.5 The threads Table"><code class="literal">threads</code></a> can be used to monitor
      activity the other thread information sources cannot.
    </p><p>
      You can always view information about your own threads. To view
      information about threads being executed for other accounts, you
      must have the <a class="link" href="security.html#priv_process"><code class="literal">PROCESS</code></a> privilege.
    </p><p>
      Each process list entry contains several pieces of information:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
          <code class="literal">Id</code> is the connection identifier for the
          client associated with the thread.
        </p></li><li class="listitem"><p>
          <code class="literal">User</code> and <code class="literal">Host</code> indicate
          the account associated with the thread.
        </p></li><li class="listitem"><p>
          <code class="literal">db</code> is the default database for the thread,
          or <code class="literal">NULL</code> if none is selected.
        </p></li><li class="listitem"><p>
          <code class="literal">Command</code> and <code class="literal">State</code>
          indicate what the thread is doing.
        </p><p>
          Most states correspond to very quick operations. If a thread
          stays in a given state for many seconds, there might be a
          problem that needs to be investigated.
        </p></li><li class="listitem"><p>
          <code class="literal">Time</code> indicates how long the thread has been
          in its current state. The thread's notion of the current
          time may be altered in some cases: The thread can change the
          time with <a class="link" href="sql-statements.html#set-variable" title="13.7.6.1 SET Syntax for Variable Assignment"><code class="literal">SET
          TIMESTAMP = <em class="replaceable"><code>value</code></em></code></a>. For a
          thread running on a slave that is processing events from the
          master, the thread time is set to the time found in the events
          and thus reflects current time on the master and not the
          slave.
        </p></li><li class="listitem"><p>
          <code class="literal">Info</code> contains the text of the statement
          being executed by the thread, or <code class="literal">NULL</code> if it
          is not executing one. By default, this value contains only the
          first 100 characters of the statement. To see the complete
          statements, use
          <a class="link" href="sql-statements.html#show-processlist" title="13.7.7.29 SHOW PROCESSLIST Statement"><code class="literal">SHOW FULL
          PROCESSLIST</code></a>.
        </p></li><li class="listitem"><p>
          The <code class="literal">sys</code> schema
          <a class="link" href="sys-schema.html#sys-processlist" title="27.4.3.22 The processlist and x$processlist Views"><code class="literal">processlist</code></a> view, which presents
          information from the Performance Schema
          <a class="link" href="performance-schema.html#threads-table" title="26.12.19.5 The threads Table"><code class="literal">threads</code></a> table in a more
          accessible format: <a class="xref" href="sys-schema.html#sys-processlist" title="27.4.3.22 The processlist and x$processlist Views">Section 27.4.3.22, “The processlist and x$processlist Views”</a>
        </p></li><li class="listitem"><p>
          The <code class="literal">sys</code> schema
          <a class="link" href="sys-schema.html#sys-session" title="27.4.3.33 The session and x$session Views"><code class="literal">session</code></a> view, which presents
          information about user sessions (like the
          <code class="literal">sys</code> schema
          <a class="link" href="sys-schema.html#sys-processlist" title="27.4.3.22 The processlist and x$processlist Views"><code class="literal">processlist</code></a> view, but with
          background processes filtered out):
          <a class="xref" href="sys-schema.html#sys-session" title="27.4.3.33 The session and x$session Views">Section 27.4.3.33, “The session and x$session Views”</a>
</p></li></ul>
</div>
<p>
      The following sections list the possible
      <code class="literal">Command</code> values, and <code class="literal">State</code>
      values grouped by category. The meaning for some of these values
      is self-evident. For others, additional description is provided.
</p>
<div class="section">

<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="thread-commands"></a>8.14.1 Thread Command Values</h3>
</div>
</div>
</div>
<a class="indexterm" name="idm46444345722160"></a><p>
        A thread can have any of the following
        <code class="literal">Command</code> values:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            <a class="indexterm" name="idm46444345719136"></a>

            <a class="indexterm" name="idm46444345717648"></a>

            <code class="literal">Binlog Dump</code>
          </p><p>
            This is a thread on a master server for sending binary log
            contents to a slave server.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345714176"></a>

            <a class="indexterm" name="idm46444345712688"></a>

            <code class="literal">Change user</code>
          </p><p>
            The thread is executing a change-user operation.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345709264"></a>

            <a class="indexterm" name="idm46444345707776"></a>

            <code class="literal">Close stmt</code>
          </p><p>
            The thread is closing a prepared statement.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345704368"></a>

            <a class="indexterm" name="idm46444345702880"></a>

            <code class="literal">Connect</code>
          </p><p>
            A replication slave is connected to its master.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345699472"></a>

            <a class="indexterm" name="idm46444345697984"></a>

            <code class="literal">Connect Out</code>
          </p><p>
            A replication slave is connecting to its master.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345694560"></a>

            <a class="indexterm" name="idm46444345693072"></a>

            <code class="literal">Create DB</code>
          </p><p>
            The thread is executing a create-database operation.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345689648"></a>

            <a class="indexterm" name="idm46444345688160"></a>

            <code class="literal">Daemon</code>
          </p><p>
            This thread is internal to the server, not a thread that
            services a client connection.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345684688"></a>

            <a class="indexterm" name="idm46444345683200"></a>

            <code class="literal">Debug</code>
          </p><p>
            The thread is generating debugging information.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345679792"></a>

            <a class="indexterm" name="idm46444345678304"></a>

            <code class="literal">Delayed insert</code>
          </p><p>
            The thread is a delayed-insert handler.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345674896"></a>

            <a class="indexterm" name="idm46444345673408"></a>

            <code class="literal">Drop DB</code>
          </p><p>
            The thread is executing a drop-database operation.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345669984"></a>

            <a class="indexterm" name="idm46444345668496"></a>

            <code class="literal">Error</code>
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345665552"></a>

            <a class="indexterm" name="idm46444345664064"></a>

            <code class="literal">Execute</code>
          </p><p>
            The thread is executing a prepared statement.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345660656"></a>

            <a class="indexterm" name="idm46444345659168"></a>

            <code class="literal">Fetch</code>
          </p><p>
            The thread is fetching the results from executing a prepared
            statement.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345655712"></a>

            <a class="indexterm" name="idm46444345654224"></a>

            <code class="literal">Field List</code>
          </p><p>
            The thread is retrieving information for table columns.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345650800"></a>

            <a class="indexterm" name="idm46444345649312"></a>

            <code class="literal">Init DB</code>
          </p><p>
            The thread is selecting a default database.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345645904"></a>

            <a class="indexterm" name="idm46444345644416"></a>

            <code class="literal">Kill</code>
          </p><p>
            The thread is killing another thread.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345641008"></a>

            <a class="indexterm" name="idm46444345639520"></a>

            <code class="literal">Long Data</code>
          </p><p>
            The thread is retrieving long data in the result of
            executing a prepared statement.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345636064"></a>

            <a class="indexterm" name="idm46444345634576"></a>

            <code class="literal">Ping</code>
          </p><p>
            The thread is handling a server-ping request.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345631168"></a>

            <a class="indexterm" name="idm46444345629680"></a>

            <code class="literal">Prepare</code>
          </p><p>
            The thread is preparing a prepared statement.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345626272"></a>

            <a class="indexterm" name="idm46444345624784"></a>

            <code class="literal">Processlist</code>
          </p><p>
            The thread is producing information about server threads.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345621360"></a>

            <a class="indexterm" name="idm46444345619872"></a>

            <code class="literal">Query</code>
          </p><p>
            The thread is executing a statement.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345616464"></a>

            <a class="indexterm" name="idm46444345614976"></a>

            <code class="literal">Quit</code>
          </p><p>
            The thread is terminating.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345611584"></a>

            <a class="indexterm" name="idm46444345610096"></a>

            <code class="literal">Refresh</code>
          </p><p>
            The thread is flushing table, logs, or caches, or resetting
            status variable or replication server information.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345606608"></a>

            <a class="indexterm" name="idm46444345605120"></a>

            <code class="literal">Register Slave</code>
          </p><p>
            The thread is registering a slave server.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345601712"></a>

            <a class="indexterm" name="idm46444345600224"></a>

            <code class="literal">Reset stmt</code>
          </p><p>
            The thread is resetting a prepared statement.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345596816"></a>

            <a class="indexterm" name="idm46444345595328"></a>

            <code class="literal">Set option</code>
          </p><p>
            The thread is setting or resetting a client
            statement-execution option.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345591872"></a>

            <a class="indexterm" name="idm46444345590384"></a>

            <code class="literal">Shutdown</code>
          </p><p>
            The thread is shutting down the server.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345586976"></a>

            <a class="indexterm" name="idm46444345585488"></a>

            <code class="literal">Sleep</code>
          </p><p>
            The thread is waiting for the client to send a new statement
            to it.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345582048"></a>

            <a class="indexterm" name="idm46444345580560"></a>

            <code class="literal">Statistics</code>
          </p><p>
            The thread is producing server-status information.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345577136"></a>

            <a class="indexterm" name="idm46444345575648"></a>

            <code class="literal">Table Dump</code>
          </p><p>
            The thread is sending table contents to a slave server.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345572224"></a>

            <a class="indexterm" name="idm46444345570736"></a>

            <code class="literal">Time</code>
          </p><p>
            Unused.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="general-thread-states"></a>8.14.2 General Thread States</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444345566512"></a><p>
        The following list describes thread <code class="literal">State</code>
        values that are associated with general query processing and not
        more specialized activities such as replication. Many of these
        are useful only for finding bugs in the server.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            <a class="indexterm" name="idm46444345562896"></a>

            <a class="indexterm" name="idm46444345561408"></a>

            <code class="literal">After create</code>
          </p><p>
            This occurs when the thread creates a table (including
            internal temporary tables), at the end of the function that
            creates the table. This state is used even if the table
            could not be created due to some error.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345557792"></a>

            <a class="indexterm" name="idm46444345556304"></a>

            <code class="literal">Analyzing</code>
          </p><p>
            The thread is calculating a <code class="literal">MyISAM</code> table
            key distributions (for example, for
            <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE TABLE</code></a>).
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345550912"></a>

            <a class="indexterm" name="idm46444345549424"></a>

            <code class="literal">checking permissions</code>
          </p><p>
            The thread is checking whether the server has the required
            privileges to execute the statement.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345545952"></a>

            <a class="indexterm" name="idm46444345544464"></a>

            <code class="literal">Checking table</code>
          </p><p>
            The thread is performing a table check operation.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345541040"></a>

            <a class="indexterm" name="idm46444345539552"></a>

            <code class="literal">cleaning up</code>
          </p><p>
            The thread has processed one command and is preparing to
            free memory and reset certain state variables.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345536064"></a>

            <a class="indexterm" name="idm46444345534576"></a>

            <code class="literal">closing tables</code>
          </p><p>
            The thread is flushing the changed table data to disk and
            closing the used tables. This should be a fast operation. If
            not, verify that you do not have a full disk and that the
            disk is not in very heavy use.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345530960"></a>

            <a class="indexterm" name="idm46444345529456"></a>

            <code class="literal">converting HEAP to ondisk</code>
          </p><p>
            The thread is converting an internal temporary table from a
            <code class="literal">MEMORY</code> table to an on-disk table.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345525264"></a>

            <a class="indexterm" name="idm46444345523776"></a>

            <code class="literal">copy to tmp table</code>
          </p><p>
            The thread is processing an <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER
            TABLE</code></a> statement. This state occurs after the table
            with the new structure has been created but before rows are
            copied into it.
          </p><p>
            For a thread in this state, the Performance Schema can be
            used to obtain about the progress of the copy operation. See
            <a class="xref" href="performance-schema.html#performance-schema-stage-tables" title="26.12.5 Performance Schema Stage Event Tables">Section 26.12.5, “Performance Schema Stage Event Tables”</a>.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345517696"></a>

            <a class="indexterm" name="idm46444345516208"></a>

            <code class="literal">Copying to group table</code>
          </p><p>
            If a statement has different <code class="literal">ORDER BY</code> and
            <code class="literal">GROUP BY</code> criteria, the rows are sorted by
            group and copied to a temporary table.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345511296"></a>

            <a class="indexterm" name="idm46444345509808"></a>

            <code class="literal">Copying to tmp table</code>
          </p><p>
            The server is copying to a temporary table in memory.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345506384"></a>

            <a class="indexterm" name="idm46444345504896"></a>

            <code class="literal">altering table</code>
          </p><p>
            The server is in the process of executing an in-place
            <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER TABLE</code></a>.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345500288"></a>

            <a class="indexterm" name="idm46444345498784"></a>

            <code class="literal">Copying to tmp table on disk</code>
          </p><p>
            The server is copying to a temporary table on disk. The
            temporary result set has become too large (see
            <a class="xref" href="optimization.html#internal-temporary-tables" title="8.4.4 Internal Temporary Table Use in MySQL">Section 8.4.4, “Internal Temporary Table Use in MySQL”</a>). Consequently,
            the thread is changing the temporary table from in-memory to
            disk-based format to save memory.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345494384"></a>

            <a class="indexterm" name="idm46444345492896"></a>

            <code class="literal">Creating index</code>
          </p><p>
            The thread is processing <code class="literal">ALTER TABLE ... ENABLE
            KEYS</code> for a <code class="literal">MyISAM</code> table.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345488112"></a>

            <a class="indexterm" name="idm46444345486624"></a>

            <code class="literal">Creating sort index</code>
          </p><p>
            The thread is processing a
            <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> that is resolved using
            an internal temporary table.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345481936"></a>

            <a class="indexterm" name="idm46444345480448"></a>

            <code class="literal">creating table</code>
          </p><p>
            The thread is creating a table. This includes creation of
            temporary tables.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345476992"></a>

            <a class="indexterm" name="idm46444345475504"></a>

            <code class="literal">Creating tmp table</code>
          </p><p>
            The thread is creating a temporary table in memory or on
            disk. If the table is created in memory but later is
            converted to an on-disk table, the state during that
            operation will be <code class="literal">Copying to tmp table on
            disk</code>.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345471184"></a>

            <a class="indexterm" name="idm46444345469664"></a>

            <code class="literal">committing alter table to storage engine</code>
          </p><p>
            The server has finished an in-place
            <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER TABLE</code></a> and is committing
            the result.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345464928"></a>

            <a class="indexterm" name="idm46444345463424"></a>

            <code class="literal">deleting from main table</code>
          </p><p>
            The server is executing the first part of a multiple-table
            delete. It is deleting only from the first table, and saving
            columns and offsets to be used for deleting from the other
            (reference) tables.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345459792"></a>

            <a class="indexterm" name="idm46444345458288"></a>

            <code class="literal">deleting from reference tables</code>
          </p><p>
            The server is executing the second part of a multiple-table
            delete and deleting the matched rows from the other tables.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345454752"></a>

            <a class="indexterm" name="idm46444345453248"></a>

            <code class="literal">discard_or_import_tablespace</code>
          </p><p>
            The thread is processing an <code class="literal">ALTER TABLE ... DISCARD
            TABLESPACE</code> or <code class="literal">ALTER TABLE ... IMPORT
            TABLESPACE</code> statement.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345448320"></a>

            <a class="indexterm" name="idm46444345446864"></a>

            <code class="literal">end</code>
          </p><p>
            This occurs at the end but before the cleanup of
            <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER TABLE</code></a>,
            <a class="link" href="sql-statements.html#create-view" title="13.1.23 CREATE VIEW Statement"><code class="literal">CREATE VIEW</code></a>,
            <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a>,
            <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>,
            <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>, or
            <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a> statements.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345436112"></a>

            <a class="indexterm" name="idm46444345434624"></a>

            <code class="literal">executing</code>
          </p><p>
            The thread has begun executing a statement.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345431216"></a>

            <a class="indexterm" name="idm46444345429712"></a>

            <code class="literal">Execution of init_command</code>
          </p><p>
            The thread is executing statements in the value of the
            <code class="literal">init_command</code> system variable.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345425520"></a>

            <a class="indexterm" name="idm46444345424032"></a>

            <code class="literal">freeing items</code>
          </p><p>
            The thread has executed a command. This state is usually
            followed by <code class="literal">cleaning up</code>.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345419872"></a>

            <a class="indexterm" name="idm46444345418384"></a>

            <code class="literal">FULLTEXT initialization</code>
          </p><p>
            The server is preparing to perform a natural-language
            full-text search.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345414928"></a>

            <a class="indexterm" name="idm46444345413440"></a>

            <code class="literal">init</code>
          </p><p>
            This occurs before the initialization of
            <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER TABLE</code></a>,
            <a class="link" href="sql-statements.html#delete" title="13.2.2 DELETE Statement"><code class="literal">DELETE</code></a>,
            <a class="link" href="sql-statements.html#insert" title="13.2.6 INSERT Statement"><code class="literal">INSERT</code></a>,
            <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>, or
            <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a> statements. Actions
            taken by the server in this state include flushing the
            binary log and the <code class="literal">InnoDB</code> log.
          </p><p>
            For the <code class="literal">end</code> state, the following
            operations could be happening:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                Writing an event to the binary log
              </p></li><li class="listitem"><p>
                Freeing memory buffers, including for blobs
</p></li></ul>
</div>
</li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345399792"></a>

            <a class="indexterm" name="idm46444345398304"></a>

            <code class="literal">Killed</code>
          </p><p>
            Someone has sent a <a class="link" href="sql-statements.html#kill" title="13.7.8.4 KILL Statement"><code class="literal">KILL</code></a>
            statement to the thread and it should abort next time it
            checks the kill flag. The flag is checked in each major loop
            in MySQL, but in some cases it might still take a short time
            for the thread to die. If the thread is locked by some other
            thread, the kill takes effect as soon as the other thread
            releases its lock.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345393312"></a>

            <a class="indexterm" name="idm46444345391824"></a>

            <code class="literal">Locking system tables</code>
          </p><p>
            The thread is trying to lock a system table (for example, a
            time zone or log table).
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345388352"></a>

            <a class="indexterm" name="idm46444345386864"></a>

            <code class="literal">logging slow query</code>
          </p><p>
            The thread is writing a statement to the slow-query log.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345383440"></a>

            <a class="indexterm" name="idm46444345381952"></a>

            <code class="literal">login</code>
          </p><p>
            The initial state for a connection thread until the client
            has been authenticated successfully.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345378480"></a>

            <a class="indexterm" name="idm46444345376992"></a>

            <code class="literal">manage keys</code>
          </p><p>
            The server is enabling or disabling a table index.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345373568"></a>

            <a class="indexterm" name="idm46444345372080"></a>

            <code class="literal">NULL</code>
          </p><p>
            This state is used for the <a class="link" href="sql-statements.html#show-processlist" title="13.7.7.29 SHOW PROCESSLIST Statement"><code class="literal">SHOW
            PROCESSLIST</code></a> state.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345367440"></a>

            <a class="indexterm" name="idm46444345365952"></a>

            <code class="literal">Opening system tables</code>
          </p><p>
            The thread is trying to open a system table (for example, a
            time zone or log table).
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345362480"></a>

            <a class="indexterm" name="idm46444345360992"></a>

            <code class="literal">Opening tables</code>
          </p><p>
            The thread is trying to open a table. This is should be very
            fast procedure, unless something prevents opening. For
            example, an <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER TABLE</code></a> or a
            <a class="link" href="sql-statements.html#lock-tables" title="13.3.6 LOCK TABLES and UNLOCK TABLES Statements"><code class="literal">LOCK
            TABLE</code></a> statement can prevent opening a table until
            the statement is finished. It is also worth checking that
            your <a class="link" href="server-administration.html#sysvar_table_open_cache"><code class="literal">table_open_cache</code></a> value
            is large enough.
          </p><p>
            For system tables, the <code class="literal">Opening system
            tables</code> state is used instead.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345352464"></a>

            <a class="indexterm" name="idm46444345350976"></a>

            <code class="literal">optimizing</code>
          </p><p>
            The server is performing initial optimizations for a query.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345347552"></a>

            <a class="indexterm" name="idm46444345346064"></a>

            <code class="literal">preparing</code>
          </p><p>
            This state occurs during query optimization.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345342656"></a>

            <a class="indexterm" name="idm46444345341168"></a>

            <code class="literal">Purging old relay logs</code>
          </p><p>
            The thread is removing unneeded relay log files.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345337744"></a>

            <a class="indexterm" name="idm46444345336256"></a>

            <code class="literal">query end</code>
          </p><p>
            This state occurs after processing a query but before the
            <code class="literal">freeing items</code> state.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345332112"></a>

            <a class="indexterm" name="idm46444345330624"></a>

            <code class="literal">Receiving from client</code>
          </p><p>
            The server is reading a packet from the client.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345327216"></a>

            <a class="indexterm" name="idm46444345325728"></a>

            <code class="literal">Removing duplicates</code>
          </p><p>
            The query was using
            <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT
            DISTINCT</code></a> in such a way that MySQL could not
            optimize away the distinct operation at an early stage.
            Because of this, MySQL requires an extra stage to remove all
            duplicated rows before sending the result to the client.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345320848"></a>

            <a class="indexterm" name="idm46444345319360"></a>

            <code class="literal">removing tmp table</code>
          </p><p>
            The thread is removing an internal temporary table after
            processing a <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a>
            statement. This state is not used if no temporary table was
            created.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345314592"></a>

            <a class="indexterm" name="idm46444345313104"></a>

            <code class="literal">rename</code>
          </p><p>
            The thread is renaming a table.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345309712"></a>

            <a class="indexterm" name="idm46444345308224"></a>

            <code class="literal">rename result table</code>
          </p><p>
            The thread is processing an <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER
            TABLE</code></a> statement, has created the new table, and is
            renaming it to replace the original table.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345303504"></a>

            <a class="indexterm" name="idm46444345302016"></a>

            <code class="literal">Reopen tables</code>
          </p><p>
            The thread got a lock for the table, but noticed after
            getting the lock that the underlying table structure
            changed. It has freed the lock, closed the table, and is
            trying to reopen it.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345298432"></a>

            <a class="indexterm" name="idm46444345296944"></a>

            <code class="literal">Repair by sorting</code>
          </p><p>
            The repair code is using a sort to create indexes.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345293520"></a>

            <a class="indexterm" name="idm46444345292016"></a>

            <code class="literal">preparing for alter table</code>
          </p><p>
            The server is preparing to execute an in-place
            <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER TABLE</code></a>.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345287328"></a>

            <a class="indexterm" name="idm46444345285840"></a>

            <code class="literal">Repair done</code>
          </p><p>
            The thread has completed a multithreaded repair for a
            <code class="literal">MyISAM</code> table.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345281696"></a>

            <a class="indexterm" name="idm46444345280208"></a>

            <code class="literal">Repair with keycache</code>
          </p><p>
            The repair code is using creating keys one by one through
            the key cache. This is much slower than <code class="literal">Repair by
            sorting</code>.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345276000"></a>

            <a class="indexterm" name="idm46444345274512"></a>

            <code class="literal">Rolling back</code>
          </p><p>
            The thread is rolling back a transaction.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345271104"></a>

            <a class="indexterm" name="idm46444345269616"></a>

            <code class="literal">Saving state</code>
          </p><p>
            For <code class="literal">MyISAM</code> table operations such as
            repair or analysis, the thread is saving the new table state
            to the <code class="filename">.MYI</code> file header. State includes
            information such as number of rows, the
            <code class="literal">AUTO_INCREMENT</code> counter, and key
            distributions.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345263888"></a>

            <a class="indexterm" name="idm46444345262384"></a>

            <code class="literal">Searching rows for update</code>
          </p><p>
            The thread is doing a first phase to find all matching rows
            before updating them. This has to be done if the
            <a class="link" href="sql-statements.html#update" title="13.2.13 UPDATE Statement"><code class="literal">UPDATE</code></a> is changing the index
            that is used to find the involved rows.
          </p></li><li class="listitem"><p>
            <code class="literal">Sending data</code>
          </p><p>
            The thread is reading and processing rows for a
            <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statement, and sending
            data to the client. Because operations occurring during this
            state tend to perform large amounts of disk access (reads),
            it is often the longest-running state over the lifetime of a
            given query.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345254144"></a>

            <a class="indexterm" name="idm46444345252656"></a>

            <code class="literal">Sending to client</code>
          </p><p>
            The server is writing a packet to the client.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345249248"></a>

            <a class="indexterm" name="idm46444345247760"></a>

            <code class="literal">setup</code>
          </p><p>
            The thread is beginning an <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER
            TABLE</code></a> operation.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345243136"></a>

            <a class="indexterm" name="idm46444345241648"></a>

            <code class="literal">Sorting for group</code>
          </p><p>
            The thread is doing a sort to satisfy a <code class="literal">GROUP
            BY</code>.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345237536"></a>

            <a class="indexterm" name="idm46444345236048"></a>

            <code class="literal">Sorting for order</code>
          </p><p>
            The thread is doing a sort to satisfy an <code class="literal">ORDER
            BY</code>.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345231936"></a>

            <a class="indexterm" name="idm46444345230448"></a>

            <code class="literal">Sorting index</code>
          </p><p>
            The thread is sorting index pages for more efficient access
            during a <code class="literal">MyISAM</code> table optimization
            operation.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345226256"></a>

            <a class="indexterm" name="idm46444345224768"></a>

            <code class="literal">Sorting result</code>
          </p><p>
            For a <a class="link" href="sql-statements.html#select" title="13.2.10 SELECT Statement"><code class="literal">SELECT</code></a> statement, this
            is similar to <code class="literal">Creating sort index</code>, but
            for nontemporary tables.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345219392"></a>

            <a class="indexterm" name="idm46444345217904"></a>

            <code class="literal">statistics</code>
          </p><p>
            The server is calculating statistics to develop a query
            execution plan. If a thread is in this state for a long
            time, the server is probably disk-bound performing other
            work.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345214320"></a>

            <a class="indexterm" name="idm46444345213248"></a>

            <a class="indexterm" name="idm46444345211760"></a>

            <a class="indexterm" name="idm46444345210272"></a>

            <code class="literal">System lock</code>
          </p><p>
            The thread has called <code class="literal">mysql_lock_tables()</code>
            and the thread state has not been updated since. This is a
            very general state that can occur for many reasons.
          </p><p>
            For example, the thread is going to request or is waiting
            for an internal or external system lock for the table. This
            can occur when <a class="link" href="innodb-storage-engine.html" title="Chapter 15 The InnoDB Storage Engine"><code class="literal">InnoDB</code></a> waits for
            a table-level lock during execution of
            <a class="link" href="sql-statements.html#lock-tables" title="13.3.6 LOCK TABLES and UNLOCK TABLES Statements"><code class="literal">LOCK TABLES</code></a>. If this state is
            being caused by requests for external locks and you are not
            using multiple <a class="link" href="programs.html#mysqld" title="4.3.1 mysqld — The MySQL Server"><span class="command"><strong>mysqld</strong></span></a> servers that are
            accessing the same <a class="link" href="storage-engines.html#myisam-storage-engine" title="16.2 The MyISAM Storage Engine"><code class="literal">MyISAM</code></a>
            tables, you can disable external system locks with the
            <a class="link" href="server-administration.html#option_mysqld_external-locking"><code class="option">--skip-external-locking</code></a>
            option. However, external locking is disabled by default, so
            it is likely that this option will have no effect. For
            <a class="link" href="sql-statements.html#show-profile" title="13.7.7.30 SHOW PROFILE Statement"><code class="literal">SHOW PROFILE</code></a>, this state
            means the thread is requesting the lock (not waiting for
            it).
          </p><p>
            For system tables, the <code class="literal">Locking system
            tables</code> state is used instead.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345196640"></a>

            <a class="indexterm" name="idm46444345195152"></a>

            <code class="literal">update</code>
          </p><p>
            The thread is getting ready to start updating the table.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345191728"></a>

            <a class="indexterm" name="idm46444345190240"></a>

            <code class="literal">Updating</code>
          </p><p>
            The thread is searching for rows to update and is updating
            them.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345186800"></a>

            <a class="indexterm" name="idm46444345185312"></a>

            <code class="literal">updating main table</code>
          </p><p>
            The server is executing the first part of a multiple-table
            update. It is updating only the first table, and saving
            columns and offsets to be used for updating the other
            (reference) tables.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345181712"></a>

            <a class="indexterm" name="idm46444345180208"></a>

            <code class="literal">updating reference tables</code>
          </p><p>
            The server is executing the second part of a multiple-table
            update and updating the matched rows from the other tables.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345176672"></a>

            <a class="indexterm" name="idm46444345175184"></a>

            <code class="literal">User lock</code>
          </p><p>
            The thread is going to request or is waiting for an advisory
            lock requested with a
            <a class="link" href="functions.html#function_get-lock"><code class="literal">GET_LOCK()</code></a> call. For
            <a class="link" href="sql-statements.html#show-profile" title="13.7.7.30 SHOW PROFILE Statement"><code class="literal">SHOW PROFILE</code></a>, this state
            means the thread is requesting the lock (not waiting for
            it).
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345169152"></a>

            <a class="indexterm" name="idm46444345167664"></a>

            <code class="literal">User sleep</code>
          </p><p>
            The thread has invoked a
            <a class="link" href="functions.html#function_sleep"><code class="literal">SLEEP()</code></a> call.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345162864"></a>

            <a class="indexterm" name="idm46444345161376"></a>

            <code class="literal">Waiting for commit lock</code>
          </p><p>
            <a class="link" href="sql-statements.html#flush-tables-with-read-lock"><code class="literal">FLUSH TABLES WITH READ LOCK</code></a>
            is waiting for a commit lock.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345156736"></a>

            <a class="indexterm" name="idm46444345155232"></a>

            <code class="literal">Waiting for handler commit</code>
          </p><p>
            The thread is waiting for a transaction to commit versus
            other parts of query processing.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345151728"></a>

            <a class="indexterm" name="idm46444345150240"></a>

            <code class="literal">Waiting for tables</code>
          </p><p>
            The thread got a notification that the underlying structure
            for a table has changed and it needs to reopen the table to
            get the new structure. However, to reopen the table, it must
            wait until all other threads have closed the table in
            question.
          </p><p>
            This notification takes place if another thread has used
            <a class="link" href="sql-statements.html#flush-tables"><code class="literal">FLUSH TABLES</code></a> or one of the
            following statements on the table in question:
            <code class="literal">FLUSH TABLES
            <em class="replaceable"><code>tbl_name</code></em></code>,
            <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER TABLE</code></a>,
            <a class="link" href="sql-statements.html#rename-table" title="13.1.36 RENAME TABLE Statement"><code class="literal">RENAME TABLE</code></a>,
            <a class="link" href="sql-statements.html#repair-table" title="13.7.3.5 REPAIR TABLE Statement"><code class="literal">REPAIR TABLE</code></a>,
            <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE TABLE</code></a>, or
            <a class="link" href="sql-statements.html#optimize-table" title="13.7.3.4 OPTIMIZE TABLE Statement"><code class="literal">OPTIMIZE TABLE</code></a>.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345137616"></a>

            <a class="indexterm" name="idm46444345136128"></a>

            <code class="literal">Waiting for table flush</code>
          </p><p>
            The thread is executing <a class="link" href="sql-statements.html#flush-tables"><code class="literal">FLUSH
            TABLES</code></a> and is waiting for all threads to close
            their tables, or the thread got a notification that the
            underlying structure for a table has changed and it needs to
            reopen the table to get the new structure. However, to
            reopen the table, it must wait until all other threads have
            closed the table in question.
          </p><p>
            This notification takes place if another thread has used
            <a class="link" href="sql-statements.html#flush-tables"><code class="literal">FLUSH TABLES</code></a> or one of the
            following statements on the table in question:
            <code class="literal">FLUSH TABLES
            <em class="replaceable"><code>tbl_name</code></em></code>,
            <a class="link" href="sql-statements.html#alter-table" title="13.1.9 ALTER TABLE Statement"><code class="literal">ALTER TABLE</code></a>,
            <a class="link" href="sql-statements.html#rename-table" title="13.1.36 RENAME TABLE Statement"><code class="literal">RENAME TABLE</code></a>,
            <a class="link" href="sql-statements.html#repair-table" title="13.7.3.5 REPAIR TABLE Statement"><code class="literal">REPAIR TABLE</code></a>,
            <a class="link" href="sql-statements.html#analyze-table" title="13.7.3.1 ANALYZE TABLE Statement"><code class="literal">ANALYZE TABLE</code></a>, or
            <a class="link" href="sql-statements.html#optimize-table" title="13.7.3.4 OPTIMIZE TABLE Statement"><code class="literal">OPTIMIZE TABLE</code></a>.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345122192"></a>

            <a class="indexterm" name="idm46444345120688"></a>

            <a class="indexterm" name="idm46444345119184"></a>

            <a class="indexterm" name="idm46444345117680"></a>

            <a class="indexterm" name="idm46444345116176"></a>

            <a class="indexterm" name="idm46444345114672"></a>

            <a class="indexterm" name="idm46444345113152"></a>

            <a class="indexterm" name="idm46444345111632"></a>

            <a class="indexterm" name="idm46444345110112"></a>

            <a class="indexterm" name="idm46444345108592"></a>

            <a class="indexterm" name="idm46444345107088"></a>

            <a class="indexterm" name="idm46444345105584"></a>

            <a class="indexterm" name="idm46444345104080"></a>

            <a class="indexterm" name="idm46444345102576"></a>

            <a class="indexterm" name="idm46444345101072"></a>

            <code class="literal">Waiting for <em class="replaceable"><code>lock_type</code></em>
            lock</code>
          </p><p>
            The server is waiting to acquire a
            <code class="literal">THR_LOCK</code> lock or a lock from the metadata
            locking subsystem, where
            <em class="replaceable"><code>lock_type</code></em> indicates the type of
            lock.
          </p><p>
            This state indicates a wait for a
            <code class="literal">THR_LOCK</code>:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">Waiting for table level lock</code>
</p></li></ul>
</div>
<p>
            These states indicate a wait for a metadata lock:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>
                <code class="literal">Waiting for event metadata lock</code>
              </p></li><li class="listitem"><p>
                <code class="literal">Waiting for global read lock</code>
              </p></li><li class="listitem"><p>
                <code class="literal">Waiting for schema metadata lock</code>
              </p></li><li class="listitem"><p>
                <code class="literal">Waiting for stored function metadata
                lock</code>
              </p></li><li class="listitem"><p>
                <code class="literal">Waiting for stored procedure metadata
                lock</code>
              </p></li><li class="listitem"><p>
                <code class="literal">Waiting for table metadata lock</code>
              </p></li><li class="listitem"><p>
                <code class="literal">Waiting for trigger metadata lock</code>
</p></li></ul>
</div>
<p>
            For information about table lock indicators, see
            <a class="xref" href="optimization.html#internal-locking" title="8.11.1 Internal Locking Methods">Section 8.11.1, “Internal Locking Methods”</a>. For information about
            metadata locking, see <a class="xref" href="optimization.html#metadata-locking" title="8.11.4 Metadata Locking">Section 8.11.4, “Metadata Locking”</a>. To
            see which locks are blocking lock requests, use the
            Performance Schema lock tables described at
            <a class="xref" href="performance-schema.html#performance-schema-lock-tables" title="26.12.13 Performance Schema Lock Tables">Section 26.12.13, “Performance Schema Lock Tables”</a>.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345078784"></a>

            <a class="indexterm" name="idm46444345077296"></a>

            <code class="literal">Waiting on cond</code>
          </p><p>
            A generic state in which the thread is waiting for a
            condition to become true. No specific state information is
            available.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345073776"></a>

            <a class="indexterm" name="idm46444345072288"></a>

            <code class="literal">Writing to net</code>
          </p><p>
            The server is writing a packet to the network.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="master-thread-states"></a>8.14.3 Replication Master Thread States</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444345067984"></a><a class="indexterm" name="idm46444345066528"></a><p>
        The following list shows the most common states you may see in
        the <code class="literal">State</code> column for the master's
        <code class="literal">Binlog Dump</code> thread. If you see no
        <code class="literal">Binlog Dump</code> threads on a master server, this
        means that replication is not running; that is, that no slaves
        are currently connected.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            <a class="indexterm" name="idm46444345061472"></a>

            <a class="indexterm" name="idm46444345059952"></a>

            <code class="literal">Finished reading one binlog; switching to next
            binlog</code>
          </p><p>
            The thread has finished reading a binary log file and is
            opening the next one to send to the slave.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345056400"></a>

            <a class="indexterm" name="idm46444345054864"></a>

            <code class="literal">Master has sent all binlog to slave; waiting for
            more updates</code>
          </p><p>
            The thread has read all remaining updates from the binary
            logs and sent them to the slave. The thread is now idle,
            waiting for new events to appear in the binary log resulting
            from new updates occurring on the master.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345051136"></a>

            <a class="indexterm" name="idm46444345049632"></a>

            <code class="literal">Sending binlog event to slave</code>
          </p><p>
            Binary logs consist of <span class="emphasis"><em>events</em></span>, where an
            event is usually an update plus some other information. The
            thread has read an event from the binary log and is now
            sending it to the slave.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345045600"></a>

            <a class="indexterm" name="idm46444345044096"></a>

            <code class="literal">Waiting to finalize termination</code>
          </p><p>
            A very brief state that occurs as the thread is stopping.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="slave-io-thread-states"></a>8.14.4 Replication Slave I/O Thread States</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444345039696"></a><a class="indexterm" name="idm46444345038240"></a><p>
        The following list shows the most common states you see in the
        <code class="literal">State</code> column for a slave server I/O thread.
        This state also appears in the <code class="literal">Slave_IO_State</code>
        column displayed by <a class="link" href="sql-statements.html#show-slave-status" title="13.7.7.34 SHOW SLAVE STATUS Statement"><code class="literal">SHOW SLAVE
        STATUS</code></a>, so you can get a good view of what is
        happening by using that statement.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            <a class="indexterm" name="idm46444345032656"></a>

            <a class="indexterm" name="idm46444345031168"></a>

            <code class="literal">Checking master version</code>
          </p><p>
            A state that occurs very briefly, after the connection to
            the master is established.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345027696"></a>

            <a class="indexterm" name="idm46444345026208"></a>

            <code class="literal">Connecting to master</code>
          </p><p>
            The thread is attempting to connect to the master.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345022784"></a>

            <a class="indexterm" name="idm46444345021280"></a>

            <code class="literal">Queueing master event to the relay log</code>
          </p><p>
            The thread has read an event and is copying it to the relay
            log so that the SQL thread can process it.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345017760"></a>

            <a class="indexterm" name="idm46444345016240"></a>

            <code class="literal">Reconnecting after a failed binlog dump
            request</code>
          </p><p>
            The thread is trying to reconnect to the master.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345012736"></a>

            <a class="indexterm" name="idm46444345011216"></a>

            <code class="literal">Reconnecting after a failed master event
            read</code>
          </p><p>
            The thread is trying to reconnect to the master. When
            connection is established again, the state becomes
            <code class="literal">Waiting for master to send event</code>.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345006912"></a>

            <a class="indexterm" name="idm46444345005408"></a>

            <code class="literal">Registering slave on master</code>
          </p><p>
            A state that occurs very briefly after the connection to the
            master is established.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444345001920"></a>

            <a class="indexterm" name="idm46444345000432"></a>

            <code class="literal">Requesting binlog dump</code>
          </p><p>
            A state that occurs very briefly, after the connection to
            the master is established. The thread sends to the master a
            request for the contents of its binary logs, starting from
            the requested binary log file name and position.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344996800"></a>

            <a class="indexterm" name="idm46444344995296"></a>

            <code class="literal">Waiting for its turn to commit</code>
          </p><p>
            A state that occurs when the slave thread is waiting for
            older worker threads to commit if
            <a class="link" href="replication.html#sysvar_slave_preserve_commit_order"><code class="literal">slave_preserve_commit_order</code></a>
            is enabled.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344990496"></a>

            <a class="indexterm" name="idm46444344988992"></a>

            <code class="literal">Waiting for master to send event</code>
          </p><p>
            The thread has connected to the master and is waiting for
            binary log events to arrive. This can last for a long time
            if the master is idle. If the wait lasts for
            <a class="link" href="replication.html#sysvar_slave_net_timeout"><code class="literal">slave_net_timeout</code></a> seconds,
            a timeout occurs. At that point, the thread considers the
            connection to be broken and makes an attempt to reconnect.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344983952"></a>

            <a class="indexterm" name="idm46444344982496"></a>

            <code class="literal">Waiting for master update</code>
          </p><p>
            The initial state before <code class="literal">Connecting to
            master</code>.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344978352"></a>

            <a class="indexterm" name="idm46444344976848"></a>

            <code class="literal">Waiting for slave mutex on exit</code>
          </p><p>
            A state that occurs briefly as the thread is stopping.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344973392"></a>

            <a class="indexterm" name="idm46444344971856"></a>

            <code class="literal">Waiting for the slave SQL thread to free enough
            relay log space</code>
          </p><p>
            You are using a nonzero
            <a class="link" href="replication.html#sysvar_relay_log_space_limit"><code class="literal">relay_log_space_limit</code></a>
            value, and the relay logs have grown large enough that their
            combined size exceeds this value. The I/O thread is waiting
            until the SQL thread frees enough space by processing relay
            log contents so that it can delete some relay log files.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344966752"></a>

            <a class="indexterm" name="idm46444344965232"></a>

            <code class="literal">Waiting to reconnect after a failed binlog dump
            request</code>
          </p><p>
            If the binary log dump request failed (due to
            disconnection), the thread goes into this state while it
            sleeps, then tries to reconnect periodically. The interval
            between retries can be specified using the
            <a class="link" href="sql-statements.html#change-master-to" title="13.4.2.1 CHANGE MASTER TO Statement"><code class="literal">CHANGE MASTER TO</code></a> statement.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344960352"></a>

            <a class="indexterm" name="idm46444344958832"></a>

            <code class="literal">Waiting to reconnect after a failed master event
            read</code>
          </p><p>
            An error occurred while reading (due to disconnection). The
            thread is sleeping for the number of seconds set by the
            <a class="link" href="sql-statements.html#change-master-to" title="13.4.2.1 CHANGE MASTER TO Statement"><code class="literal">CHANGE MASTER TO</code></a> statement
            (default 60) before attempting to reconnect.
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="slave-sql-thread-states"></a>8.14.5 Replication Slave SQL Thread States</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444344953056"></a><a class="indexterm" name="idm46444344951600"></a><p>
        The following list shows the most common states you may see in
        the <code class="literal">State</code> column for a slave server SQL
        thread:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            <a class="indexterm" name="idm46444344948112"></a>

            <a class="indexterm" name="idm46444344946624"></a>

            <code class="literal">Killing slave</code>
          </p><p>
            The thread is processing a <code class="literal">STOP SLAVE</code>
            statement.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344942496"></a>

            <a class="indexterm" name="idm46444344940960"></a>

            <code class="literal">Making temporary file (append) before replaying
            LOAD DATA INFILE</code>
          </p><p>
            The thread is executing a <a class="link" href="sql-statements.html#load-data" title="13.2.7 LOAD DATA Statement"><code class="literal">LOAD
            DATA</code></a> statement and is appending the data to a
            temporary file containing the data from which the slave will
            read rows.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344936112"></a>

            <a class="indexterm" name="idm46444344934576"></a>

            <code class="literal">Making temporary file (create) before replaying
            LOAD DATA INFILE</code>
          </p><p>
            The thread is executing a <a class="link" href="sql-statements.html#load-data" title="13.2.7 LOAD DATA Statement"><code class="literal">LOAD
            DATA</code></a> statement and is creating a temporary file
            containing the data from which the slave will read rows.
            This state can only be encountered if the original
            <a class="link" href="sql-statements.html#load-data" title="13.2.7 LOAD DATA Statement"><code class="literal">LOAD DATA</code></a> statement was
            logged by a master running a version of MySQL lower than
            MySQL 5.0.3.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344928352"></a>

            <a class="indexterm" name="idm46444344926848"></a>

            <code class="literal">Reading event from the relay log</code>
          </p><p>
            The thread has read an event from the relay log so that the
            event can be processed.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344923360"></a>

            <a class="indexterm" name="idm46444344921840"></a>

            <code class="literal">Slave has read all relay log; waiting for more
            updates</code>
          </p><p>
            The thread has processed all events in the relay log files,
            and is now waiting for the I/O thread to write new events to
            the relay log.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344918240"></a>

            <a class="indexterm" name="idm46444344916736"></a>

            <code class="literal">Waiting for an event from Coordinator</code>
          </p><p>
            Using the multithreaded slave
            (<a class="link" href="replication.html#sysvar_slave_parallel_workers"><code class="literal">slave_parallel_workers</code></a> is
            greater than 1), one of the slave worker threads is waiting
            for an event from the coordinator thread.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344911856"></a>

            <a class="indexterm" name="idm46444344910400"></a>

            <code class="literal">Waiting for slave mutex on exit</code>
          </p><p>
            A very brief state that occurs as the thread is stopping.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344906944"></a>

            <a class="indexterm" name="idm46444344905424"></a>

            <code class="literal">Waiting for Slave Workers to free pending
            events</code>
          </p><p>
            This waiting action occurs when the total size of events
            being processed by Workers exceeds the size of the
            <a class="link" href="replication.html#sysvar_slave_pending_jobs_size_max"><code class="literal">slave_pending_jobs_size_max</code></a>
            system variable. The Coordinator resumes scheduling when the
            size drops below this limit. This state occurs only when
            <a class="link" href="replication.html#sysvar_slave_parallel_workers"><code class="literal">slave_parallel_workers</code></a> is
            set greater than 0.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344899088"></a>

            <a class="indexterm" name="idm46444344897632"></a>

            <code class="literal">Waiting for the next event in relay log</code>
          </p><p>
            The initial state before <code class="literal">Reading event from the
            relay log</code>.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344893472"></a>

            <a class="indexterm" name="idm46444344891936"></a>

            <code class="literal">Waiting until MASTER_DELAY seconds after master
            executed event</code>
          </p><p>
            The SQL thread has read an event but is waiting for the
            slave delay to lapse. This delay is set with the
            <code class="literal">MASTER_DELAY</code> option of
            <a class="link" href="sql-statements.html#change-master-to" title="13.4.2.1 CHANGE MASTER TO Statement"><code class="literal">CHANGE MASTER TO</code></a>.
</p></li></ul>
</div>
<p>
        The <code class="literal">Info</code> column for the SQL thread may also
        show the text of a statement. This indicates that the thread has
        read an event from the relay log, extracted the statement from
        it, and may be executing it.
</p>
</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="slave-connection-thread-states"></a>8.14.6 Replication Slave Connection Thread States</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444344884032"></a><a class="indexterm" name="idm46444344882544"></a><p>
        These thread states occur on a replication slave but are
        associated with connection threads, not with the I/O or SQL
        threads.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            <a class="indexterm" name="idm46444344878864"></a>

            <a class="indexterm" name="idm46444344877376"></a>

            <code class="literal">Changing master</code>
          </p><p>
            The thread is processing a <a class="link" href="sql-statements.html#change-master-to" title="13.4.2.1 CHANGE MASTER TO Statement"><code class="literal">CHANGE
            MASTER TO</code></a> statement.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344872784"></a>

            <a class="indexterm" name="idm46444344871296"></a>

            <code class="literal">Killing slave</code>
          </p><p>
            The thread is processing a <code class="literal">STOP SLAVE</code>
            statement.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344867168"></a>

            <a class="indexterm" name="idm46444344865664"></a>

            <code class="literal">Opening master dump table</code>
          </p><p>
            This state occurs after <code class="literal">Creating table from master
            dump</code>.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344861504"></a>

            <a class="indexterm" name="idm46444344860000"></a>

            <code class="literal">Reading master dump table data</code>
          </p><p>
            This state occurs after <code class="literal">Opening master dump
            table</code>.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344855856"></a>

            <a class="indexterm" name="idm46444344854336"></a>

            <code class="literal">Rebuilding the index on master dump table</code>
          </p><p>
            This state occurs after <code class="literal">Reading master dump table
            data</code>.
</p></li></ul>
</div>

</div>

<div class="section">

<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="mysql-cluster-thread-states"></a>8.14.7 NDB Cluster Thread States</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444344849200"></a><a class="indexterm" name="idm46444344847712"></a>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            <a class="indexterm" name="idm46444344844656"></a>

            <a class="indexterm" name="idm46444344843152"></a>

            <code class="literal">Committing events to binlog</code>
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344840176"></a>

            <a class="indexterm" name="idm46444344838672"></a>

            <code class="literal">Opening mysql.ndb_apply_status</code>
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344835696"></a>

            <a class="indexterm" name="idm46444344834208"></a>

            <code class="literal">Processing events</code>
          </p><p>
            The thread is processing events for binary logging.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344830784"></a>

            <a class="indexterm" name="idm46444344829280"></a>

            <code class="literal">Processing events from schema table</code>
          </p><p>
            The thread is doing the work of schema replication.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344825824"></a>

            <a class="indexterm" name="idm46444344824336"></a>

            <code class="literal">Shutting down</code>
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344821392"></a>

            <a class="indexterm" name="idm46444344819872"></a>

            <code class="literal">Syncing ndb table schema operation and
            binlog</code>
          </p><p>
            This is used to have a correct binary log of schema
            operations for NDB.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344816336"></a>

            <a class="indexterm" name="idm46444344814800"></a>

            <code class="literal">Waiting for allowed to take ndbcluster global
            schema lock</code>
          </p><p>
            The thread is waiting for permission to take a global schema
            lock.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344811264"></a>

            <a class="indexterm" name="idm46444344809760"></a>

            <code class="literal">Waiting for event from ndbcluster</code>
          </p><p>
            The server is acting as an SQL node in an NDB Cluster, and
            is connected to a cluster management node.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344806240"></a>

            <a class="indexterm" name="idm46444344804736"></a>

            <code class="literal">Waiting for first event from ndbcluster</code>
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344801760"></a>

            <a class="indexterm" name="idm46444344800224"></a>

            <code class="literal">Waiting for ndbcluster binlog update to reach
            current position</code>
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344797168"></a>

            <a class="indexterm" name="idm46444344795648"></a>

            <code class="literal">Waiting for ndbcluster global schema lock</code>
          </p><p>
            The thread is waiting for a global schema lock held by
            another thread to be released.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344792112"></a>

            <a class="indexterm" name="idm46444344790608"></a>

            <code class="literal">Waiting for ndbcluster to start</code>
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344787632"></a>

            <a class="indexterm" name="idm46444344786128"></a>

            <code class="literal">Waiting for schema epoch</code>
          </p><p>
            The thread is waiting for a schema epoch (that is, a global
            checkpoint).
</p></li></ul>
</div>

</div>
<div class="section">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a name="event-scheduler-thread-states"></a>8.14.8 Event Scheduler Thread States</h3>

</div>

</div>

</div>
<a class="indexterm" name="idm46444344781760"></a><a class="indexterm" name="idm46444344780272"></a><p>
        These states occur for the Event Scheduler thread, threads that
        are created to execute scheduled events, or threads that
        terminate the scheduler.
</p>
<div class="itemizedlist">
<ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>
            <a class="indexterm" name="idm46444344777440"></a>

            <a class="indexterm" name="idm46444344775952"></a>

            <code class="literal">Clearing</code>
          </p><p>
            The scheduler thread or a thread that was executing an event
            is terminating and is about to end.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344772480"></a>

            <a class="indexterm" name="idm46444344770992"></a>

            <code class="literal">Initialized</code>
          </p><p>
            The scheduler thread or a thread that will execute an event
            has been initialized.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344767536"></a>

            <a class="indexterm" name="idm46444344766032"></a>

            <code class="literal">Waiting for next activation</code>
          </p><p>
            The scheduler has a nonempty event queue but the next
            activation is in the future.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344762544"></a>

            <a class="indexterm" name="idm46444344761040"></a>

            <code class="literal">Waiting for scheduler to stop</code>
          </p><p>
            The thread issued <code class="literal">SET GLOBAL
            event_scheduler=OFF</code> and is waiting for the
            scheduler to stop.
          </p></li><li class="listitem"><p>
            <a class="indexterm" name="idm46444344756832"></a>

            <a class="indexterm" name="idm46444344755344"></a>

            <code class="literal">Waiting on empty queue</code>
          </p><p>
            The scheduler's event queue is empty and it is
            sleeping.
</p></li></ul>
</div>

</div>

</div>

</div>
<div class="copyright-footer">

</div>
<div class="navfooter">
<hr>
<table width="100%" summary="Navigation footer">
<tr>
<td width="40%" align="left"><a accesskey="p" href="backup-and-recovery.html">Prev</a></td>
<td width="20%" align="center"><a accesskey="u" href="">Up</a></td>
<td width="40%" align="right"> <a accesskey="n" href="language-structure.html">Next</a></td>
</tr>
<tr>
<td width="40%" align="left" valign="top">Chapter 7 Backup and Recovery</td>
<td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td>
<td width="40%" align="right" valign="top">Chapter 9 Language Structure</td>
</tr>
</table>
</div>
</body>
</html>
